Isaca AAIA Practice Test 4
Question 30
An independent AI auditor reviews this case: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
- Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Explanation
Correct Answers
B:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. The corrective action is: Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
D:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. For this situation: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
Incorrect Answers
A:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. For this situation: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
C:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
E:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
Question 31
A governance and assurance review uncovers the following: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
- Examine decision map, error costs, affected populations, human override authority, complaints and impact assessment.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
Explanation
Correct Answer
A:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. The acceptable verified outcome is: Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
Incorrect Answers
B:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
C:This option concerns Predictive versus generative model task fit, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
D:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
Question 32
A risk-based audit engagement examines this operating condition: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Sampled denials have faithful understandable reasons, reliable escalation and documented accuracy versus simpler alternatives.
- Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
- Expired or high-risk exceptions are renewed through accountable approval or removed with verified closure.
Explanation
Correct Answer
C:This is an observable validation result for Unsupervised anomaly detection and alert governance, not just management assurance: Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
Incorrect Answers
A:For this situation: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
B:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. For this situation: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
D:This option concerns AI policy exception governance, which is a related but different AI governance concern. For this situation: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
Question 33
An audit committee asks for a defensible assessment of this situation: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Design usable human review with understandable context, authority to intervene, escalation and traceable decisions.
- Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
Explanation
Correct Answer
A:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
Incorrect Answers
B:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
C:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
D:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
Question 34
During an AI controls assessment, the team learns that a security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Governance exceptions are controlled risk decisions rather than silent removal of mandated requirements.
- Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
- An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
Explanation
Correct Answer
C:Independent audit evidence includes: Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
Incorrect Answers
A:This option concerns AI policy exception governance, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
B:This option concerns Predictive versus generative model task fit, which is a related but different AI governance concern. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
D:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. For this situation: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
Question 35
An independent AI auditor reviews this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Inspect use-case criticality, model comparison, reason-code fidelity, feature lineage, adverse-decision samples and appeal test results.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
Explanation
Correct Answer
A:In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
Incorrect Answers
B:This option concerns Predictive versus generative model task fit, which is a related but different AI governance concern. For this situation: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
C:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
D:For this situation: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
Question 36
A governance and assurance review uncovers the following: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Adverse outcomes, challenge rights and corrections are monitored with approved accountable decision owners.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
- A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Explanation
Correct Answer
C:Independent audit evidence includes: Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
Incorrect Answers
A:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. In this case: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
B:It misses the key control principle: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
D:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. For this situation: A security team cannot explain why a new unsupervised clustering system groups seasonal customer behavior as suspicious.
Question 37
A risk-based audit engagement examines this operating condition: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- Evaluate shared data, dependencies and failure modes of model components, rather than counting models as independent assurance.
Explanation
Correct Answer
B:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. In this case: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
Incorrect Answers
A:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. It does not establish control effectiveness for Retrieval-augmented answer grounding.
C:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document.
D:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. It does not establish control effectiveness for Retrieval-augmented answer grounding.
Question 38
An audit committee asks for a defensible assessment of this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
- Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
- Model diversity does not eliminate common-cause errors when inputs, architectures or suppliers are correlated.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
Explanation
Correct Answer
B:The corrective action is: Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference. In this case: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
Incorrect Answers
A:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. For this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
C:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. For this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
D:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. For this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
Question 39
During an AI controls assessment, the team learns that a legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
- Trace shared feature pipelines, benchmark provenance, model supplier relationships, correlated misses and failover tests.
- Sample user interface, override permissions, time pressure, rationale records and actual reviewer interventions.
Explanation
Correct Answer
B:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. The corrective action is: Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
Incorrect Answers
A:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. The auditor should instead examine: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
C:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. The decisive principle here is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
D:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. It does not establish control effectiveness for Retrieval-augmented answer grounding.
Question 40
An independent AI auditor reviews this case: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- Sample user interface, override permissions, time pressure, rationale records and actual reviewer interventions.
Explanation
Correct Answers
B:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. For this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
C:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. This evidence directly tests the asserted control for Retrieval-augmented answer grounding: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Incorrect Answers
A:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. The decisive principle here is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
D:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
E:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. For this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
