Isaca AAIA Practice Test 3
Question 20
An independent AI auditor reviews this case: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Trace shared feature pipelines, benchmark provenance, model supplier relationships, correlated misses and failover tests.
- Assess individual and group impact before deployment, require proportional human intervention and record appeal and remediation processes.
Explanation
Correct Answers
A:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
C:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. The acceptable verified outcome is: Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
Incorrect Answers
B:For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
D:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
E:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
Question 21
A governance and assurance review uncovers the following: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate shared data, dependencies and failure modes of model components, rather than counting models as independent assurance.
- Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
Explanation
Correct Answer
B:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. In this case: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
Incorrect Answers
A:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. In this case: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
C:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. In this case: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
D:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. It does not establish control effectiveness for Predictive versus generative model task fit.
Question 22
A risk-based audit engagement examines this operating condition: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Privacy by design changes processing architecture rather than relying solely on downstream access policy.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
- Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
Explanation
Correct Answer
D:The corrective action is: Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review. In this case: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
Incorrect Answers
A:This option concerns Privacy by design for model features, which is a related but different AI governance concern. In this case: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
B:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. For this situation: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
C:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. For this situation: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
Question 23
An audit committee asks for a defensible assessment of this situation: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Compare feature necessity, identifiability, model utility, transformation design and data retention.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
Explanation
Correct Answer
C:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. Independent audit evidence includes: Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
Incorrect Answers
A:For this situation: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
B:This option concerns Privacy by design for model features, which is a related but different AI governance concern. For this situation: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
D:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
Question 24
During an AI controls assessment, the team learns that a finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Independent challenge tests detect correlated failure and evidence supports any reliability uplift claimed for the ensemble.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
- A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Explanation
Correct Answer
C:This is an observable validation result for Predictive versus generative model task fit, not just management assurance: Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
Incorrect Answers
A:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. For this situation: A finance team asks a large language model to replace its deterministic tax-rate calculation service because it sounds more natural.
B:It misses the key control principle: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
D:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
Question 25
An independent AI auditor reviews this case: An anti-fraud team treats every unsupervised anomaly score as proof of an illegal transaction and freezes accounts automatically. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
- Evaluate shared data, dependencies and failure modes of model components, rather than counting models as independent assurance.
Explanation
Correct Answer
B:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. Independent audit evidence includes: Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
Incorrect Answers
A:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
C:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. It does not establish control effectiveness for Unsupervised anomaly detection and alert governance.
D:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. It does not establish control effectiveness for Unsupervised anomaly detection and alert governance.
Question 26
A governance and assurance review uncovers the following: An anti-fraud team treats every unsupervised anomaly score as proof of an illegal transaction and freezes accounts automatically. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Model diversity does not eliminate common-cause errors when inputs, architectures or suppliers are correlated.
- An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
- A human rubber stamp without information or authority does not provide effective human oversight.
Explanation
Correct Answer
C:Independent audit evidence includes: Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications. This is the applicable AI audit principle for Unsupervised anomaly detection and alert governance: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
Incorrect Answers
A:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. For this situation: An anti-fraud team treats every unsupervised anomaly score as proof of an illegal transaction and freezes accounts automatically.
B:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
D:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. It does not establish control effectiveness for Unsupervised anomaly detection and alert governance.
Question 27
A risk-based audit engagement examines this operating condition: An anti-fraud team treats every unsupervised anomaly score as proof of an illegal transaction and freezes accounts automatically. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
Explanation
Correct Answer
D:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. This evidence directly tests the asserted control for Unsupervised anomaly detection and alert governance: Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
Incorrect Answers
A:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. It does not establish control effectiveness for Unsupervised anomaly detection and alert governance.
B:This option concerns Predictive versus generative model task fit, which is a related but different AI governance concern. It does not establish control effectiveness for Unsupervised anomaly detection and alert governance.
C:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. The auditor should instead examine: Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
Question 28
An audit committee asks for a defensible assessment of this situation: An anti-fraud team treats every unsupervised anomaly score as proof of an illegal transaction and freezes accounts automatically. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Reviewers demonstrate independent judgment and appropriate overturns with auditable documentation.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Adverse outcomes, challenge rights and corrections are monitored with approved accountable decision owners.
- Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
Explanation
Correct Answer
D:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. This is an observable validation result for Unsupervised anomaly detection and alert governance, not just management assurance: Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
Incorrect Answers
A:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
B:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. Effective closure requires: Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
C:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. It does not establish control effectiveness for Unsupervised anomaly detection and alert governance.
Question 29
During an AI controls assessment, the team learns that a manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Require time-bound exception approvals, explicit risk ownership, compensating measures and periodic review.
- Evaluate shared data, dependencies and failure modes of model components, rather than counting models as independent assurance.
Explanation
Correct Answer
B:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. In this case: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
Incorrect Answers
A:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. In this case: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
C:This option concerns AI policy exception governance, which is a related but different AI governance concern. It does not establish control effectiveness for Unsupervised anomaly detection and alert governance.
D:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. In this case: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
Question 30
An independent AI auditor reviews this case: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
- Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Explanation
Correct Answers
B:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. The corrective action is: Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
D:The reason it is necessary is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion. For this situation: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
Incorrect Answers
A:This attractive shortcut does not resolve the Unsupervised anomaly detection and alert governance weakness: Use raw anomaly rank alone to make irreversible punitive decisions without independent evidence. For this situation: A manufacturer detects novel sensor deviations but its model assumes a normal distribution that excludes weekend maintenance.
C:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
E:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. The decisive principle here is: An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
