Isaca AAIA Practice Test 78
Question 770
An independent AI auditor reviews this case: A public chatbot is tested only for friendly prompts while attackers can upload documents with conflicting instructions. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
- Conduct scoped adversarial testing across inputs, retrieved context, tools, identities and post-change regressions.
- Inspect source passage matches, structured evaluation rubrics, omission rates, unsupported-claim samples and reviewer disagreements.
- Load-test realistic context sizes, concurrency, cost limits, percentiles and fallback behavior.
- Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
Explanation
Correct Answers
B:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. For this situation: A public chatbot is tested only for friendly prompts while attackers can upload documents with conflicting instructions.
E:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. This evidence directly tests the asserted control for AI adversarial red-team scope and regression: Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
Incorrect Answers
A:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. For this situation: A public chatbot is tested only for friendly prompts while attackers can upload documents with conflicting instructions.
C:This option concerns Grounded generative answer quality evaluation, which is a related but different AI governance concern. It does not establish control effectiveness for AI adversarial red-team scope and regression.
D:This option concerns AI service latency cost and capacity testing, which is a related but different AI governance concern. It does not establish control effectiveness for AI adversarial red-team scope and regression.
Question 771
A governance and assurance review uncovers the following: A public chatbot is tested only for friendly prompts while attackers can upload documents with conflicting instructions. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
- Inspect stratified confusion matrices, sample sizes, relevant context, confidence intervals and exception responses.
- Inspect segment coverage, stress datasets, input validation, abstention, drift triggers and error patterns.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
Explanation
Correct Answer
A:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. The corrective action is: Conduct scoped adversarial testing across inputs, retrieved context, tools, identities and post-change regressions.
Incorrect Answers
B:This option concerns Subgroup fairness performance validation, which is a related but different AI governance concern. It does not establish control effectiveness for AI adversarial red-team scope and regression.
C:This option concerns Out-of-distribution robustness and shift testing, which is a related but different AI governance concern. It does not establish control effectiveness for AI adversarial red-team scope and regression.
D:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. The auditor should instead examine: Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
Question 772
A risk-based audit engagement examines this operating condition: A public chatbot is tested only for friendly prompts while attackers can upload documents with conflicting instructions. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Critical attack paths are blocked by tested technical controls and regressions catch recurrence.
- Performance advantage remains on independent representative evaluation with minimal leakage.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
- Approved subgroup criteria are met or transparent residual disparities receive treatment and escalation.
Explanation
Correct Answer
A:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. This is an observable validation result for AI adversarial red-team scope and regression, not just management assurance: Critical attack paths are blocked by tested technical controls and regressions catch recurrence.
Incorrect Answers
B:This option concerns Training and evaluation benchmark contamination, which is a related but different AI governance concern. For this situation: A public chatbot is tested only for friendly prompts while attackers can upload documents with conflicting instructions.
C:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. Effective closure requires: Critical attack paths are blocked by tested technical controls and regressions catch recurrence.
D:This option concerns Subgroup fairness performance validation, which is a related but different AI governance concern. For this situation: A public chatbot is tested only for friendly prompts while attackers can upload documents with conflicting instructions.
Question 773
An audit committee asks for a defensible assessment of this situation: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate behavior under credible distribution shifts and reject or safely defer unsupported inputs.
- Conduct scoped adversarial testing across inputs, retrieved context, tools, identities and post-change regressions.
- Evaluate justified subgroup error and outcome metrics with adequate samples and documented mitigation thresholds.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
Explanation
Correct Answer
B:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. In this case: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection.
Incorrect Answers
A:This option concerns Out-of-distribution robustness and shift testing, which is a related but different AI governance concern. The decisive principle here is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
C:This option concerns Subgroup fairness performance validation, which is a related but different AI governance concern. The decisive principle here is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
D:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. In this case: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection.
Question 774
During an AI controls assessment, the team learns that a payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Fluency and coherence are not evidence of factual correctness or fidelity to source material.
- AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
- AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
Explanation
Correct Answer
B:Independent audit evidence includes: Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence. This is the applicable AI audit principle for AI adversarial red-team scope and regression: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
Incorrect Answers
A:This option concerns Grounded generative answer quality evaluation, which is a related but different AI governance concern. The decisive principle here is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The decisive principle here is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
D:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. For this situation: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection.
Question 775
An independent AI auditor reviews this case: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect training/evaluation overlap, duplicate hashes, benchmark dates, partition strategy and blind results.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
- Inspect source passage matches, structured evaluation rubrics, omission rates, unsupported-claim samples and reviewer disagreements.
- Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
Explanation
Correct Answer
D:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. The acceptable verified outcome is: Critical attack paths are blocked by tested technical controls and regressions catch recurrence.
Incorrect Answers
A:This option concerns Training and evaluation benchmark contamination, which is a related but different AI governance concern. The decisive principle here is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
B:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. Conduct scoped adversarial testing across inputs, retrieved context, tools, identities and post-change regressions.
C:This option concerns Grounded generative answer quality evaluation, which is a related but different AI governance concern. For this situation: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection.
Question 776
A governance and assurance review uncovers the following: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Critical attack paths are blocked by tested technical controls and regressions catch recurrence.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
- Independent reviewers confirm supported statements and absence of fabricated references within risk tolerances.
- Approved subgroup criteria are met or transparent residual disparities receive treatment and escalation.
Explanation
Correct Answer
A:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. Independent audit evidence includes: Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
Incorrect Answers
B:For this situation: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection.
C:This option concerns Grounded generative answer quality evaluation, which is a related but different AI governance concern. The audit must corroborate: Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
D:This option concerns Subgroup fairness performance validation, which is a related but different AI governance concern. For this situation: A payment agent passes accuracy tests but no one challenges tool-call escalation or indirect prompt injection.
Question 777
A risk-based audit engagement examines this operating condition: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
- Conduct scoped adversarial testing across inputs, retrieved context, tools, identities and post-change regressions.
- Evaluate justified subgroup error and outcome metrics with adequate samples and documented mitigation thresholds.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
Explanation
Correct Answer
B:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
Incorrect Answers
A:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
C:This option concerns Subgroup fairness performance validation, which is a related but different AI governance concern. In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
D:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
Question 778
An audit committee asks for a defensible assessment of this situation: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Averages hide tail failures and resource exhaustion; service acceptability includes efficiency and availability.
- AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
- Benchmark scores can overstate generalization when test items or related records have been exposed to development.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
Explanation
Correct Answer
B:Independent audit evidence includes: Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence. In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
Incorrect Answers
A:This option concerns AI service latency cost and capacity testing, which is a related but different AI governance concern. The decisive principle here is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
C:This option concerns Training and evaluation benchmark contamination, which is a related but different AI governance concern. In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
D:This attractive shortcut does not resolve the AI adversarial red-team scope and regression weakness: Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment. For this situation: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
Question 779
During an AI controls assessment, the team learns that a legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
- Inspect p95/p99 response, token counts, queue depth, error budgets, concurrency and billing ceilings.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
Explanation
Correct Answer
A:In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
Incorrect Answers
B:This option concerns AI service latency cost and capacity testing, which is a related but different AI governance concern. For this situation: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. For this situation: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
D:For this situation: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
Question 780
An independent AI auditor reviews this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Call one successful refusal to an obvious unsafe prompt a comprehensive security red-team assessment.
- Inspect threat model, attack corpus diversity, tool permissions, exploit transcripts and remediation evidence.
- Test citation support, omissions, contradiction handling and safe abstention using independently labeled examples.
- Conduct scoped adversarial testing across inputs, retrieved context, tools, identities and post-change regressions.
- Inspect stratified confusion matrices, sample sizes, relevant context, confidence intervals and exception responses.
Explanation
Correct Answers
B:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. For this situation: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
D:The reason it is necessary is: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases. This response fits the risk associated with AI adversarial red-team scope and regression: Conduct scoped adversarial testing across inputs, retrieved context, tools, identities and post-change regressions.
Incorrect Answers
A:It misses the key control principle: AI misuse testing requires plausible attacker-controlled surfaces and repeatable negative cases.
C:This option concerns Grounded generative answer quality evaluation, which is a related but different AI governance concern. In this case: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
E:This option concerns Subgroup fairness performance validation, which is a related but different AI governance concern. For this situation: A legal assistant detects one jailbreak phrase but fails paraphrased attempts after a prompt update.
