Isaca AAIA Practice Test 2
Question 10
An independent AI auditor reviews this case: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect use-case criticality, model comparison, reason-code fidelity, feature lineage, adverse-decision samples and appeal test results.
- Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Approve the highest aggregate AUC and assume a supplier marketing statement proves individual decisions are explainable.
Explanation
Correct Answers
A:The reason it is necessary is: Model suitability depends on performance, explainability and consequences of errors; a marginal metric gain does not remove oversight obligations. The corrective action is: Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
B:The reason it is necessary is: Model suitability depends on performance, explainability and consequences of errors; a marginal metric gain does not remove oversight obligations. For this situation: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
Incorrect Answers
C:This option concerns Retrieval-augmented answer grounding, which is a related but different AI governance concern. In this case: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
D:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. In this case: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
E:This attractive shortcut does not resolve the Explainable credit-model selection for adverse decisions weakness: Approve the highest aggregate AUC and assume a supplier marketing statement proves individual decisions are explainable. For this situation: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
Question 11
A governance and assurance review uncovers the following: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Examine decision map, error costs, affected populations, human override authority, complaints and impact assessment.
- Inspect use-case criticality, model comparison, reason-code fidelity, feature lineage, adverse-decision samples and appeal test results.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Approve the highest aggregate AUC and assume a supplier marketing statement proves individual decisions are explainable.
Explanation
Correct Answer
B:For this situation: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
Incorrect Answers
A:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. For this situation: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
C:This option concerns Predictive versus generative model task fit, which is a related but different AI governance concern. It does not establish control effectiveness for Explainable credit-model selection for adverse decisions.
D:It misses the key control principle: Model suitability depends on performance, explainability and consequences of errors; a marginal metric gain does not remove oversight obligations.
Question 12
A risk-based audit engagement examines this operating condition: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
- Sampled denials have faithful understandable reasons, reliable escalation and documented accuracy versus simpler alternatives.
- Approve the highest aggregate AUC and assume a supplier marketing statement proves individual decisions are explainable.
- Validated reduced-data features meet model purpose without exposing unnecessary identities or content.
Explanation
Correct Answer
B:This is an observable validation result for Explainable credit-model selection for adverse decisions, not just management assurance: Sampled denials have faithful understandable reasons, reliable escalation and documented accuracy versus simpler alternatives.
Incorrect Answers
A:This option concerns Predictive versus generative model task fit, which is a related but different AI governance concern. For this situation: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
C:The auditor should instead examine: Inspect use-case criticality, model comparison, reason-code fidelity, feature lineage, adverse-decision samples and appeal test results.
D:This option concerns Privacy by design for model features, which is a related but different AI governance concern. For this situation: A regional bank wants automatic credit limits from a vendor model and has no access to feature importance or decision-level reason codes.
Question 13
An audit committee asks for a defensible assessment of this situation: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate shared data, dependencies and failure modes of model components, rather than counting models as independent assurance.
- Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review.
- Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
Explanation
Correct Answer
B:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. In this case: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations.
Incorrect Answers
A:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. It does not establish control effectiveness for Predictive versus generative model task fit.
C:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. It does not establish control effectiveness for Predictive versus generative model task fit.
D:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
Question 14
During an AI controls assessment, the team learns that an insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- A useful aggregate prediction may still cause disproportionate consequences for affected people and demands context-specific governance.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
- Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
Explanation
Correct Answer
D:The corrective action is: Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review. In this case: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations.
Incorrect Answers
A:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. It does not establish control effectiveness for Predictive versus generative model task fit.
B:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. For this situation: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations.
C:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. It does not establish control effectiveness for Predictive versus generative model task fit.
Question 15
An independent AI auditor reviews this case: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Sample user interface, override permissions, time pressure, rationale records and actual reviewer interventions.
- Compare feature necessity, identifiability, model utility, transformation design and data retention.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
Explanation
Correct Answer
D:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. This evidence directly tests the asserted control for Predictive versus generative model task fit: Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
Incorrect Answers
A:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. It does not establish control effectiveness for Predictive versus generative model task fit.
B:This option concerns Privacy by design for model features, which is a related but different AI governance concern. It does not establish control effectiveness for Predictive versus generative model task fit.
C:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. The auditor should instead examine: Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
Question 16
A governance and assurance review uncovers the following: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
- Adverse outcomes, challenge rights and corrections are monitored with approved accountable decision owners.
Explanation
Correct Answer
C:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. This is an observable validation result for Predictive versus generative model task fit, not just management assurance: Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
Incorrect Answers
A:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. For this situation: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations.
B:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review.
D:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. For this situation: An insurer commissions a text-generating assistant to calculate actuarial reserve numbers without numerical controls or reconciliations.
Question 17
A risk-based audit engagement examines this operating condition: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Test whether less identifying features or on-premises transformations satisfy the business purpose.
- Design usable human review with understandable context, authority to intervene, escalation and traceable decisions.
Explanation
Correct Answer
A:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. In this case: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
Incorrect Answers
B:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. In this case: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
C:This option concerns Privacy by design for model features, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
D:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
Question 18
An audit committee asks for a defensible assessment of this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Model suitability depends on performance, explainability and consequences of errors; a marginal metric gain does not remove oversight obligations.
- Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
- Privacy by design changes processing architecture rather than relying solely on downstream access policy.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
Explanation
Correct Answer
B:The corrective action is: Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review. In this case: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
Incorrect Answers
A:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
C:This option concerns Privacy by design for model features, which is a related but different AI governance concern. For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
D:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
Question 19
During an AI controls assessment, the team learns that a call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Trace shared feature pipelines, benchmark provenance, model supplier relationships, correlated misses and failover tests.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Compare feature necessity, identifiability, model utility, transformation design and data retention.
Explanation
Correct Answer
B:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. The corrective action is: Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review.
Incorrect Answers
A:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
C:This attractive shortcut does not resolve the Predictive versus generative model task fit weakness: Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types. Effective closure requires: Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
D:This option concerns Privacy by design for model features, which is a related but different AI governance concern. The audit must corroborate: Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
Question 20
An independent AI auditor reviews this case: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Match the model family to the task, retain deterministic calculations where appropriate and validate generative outputs against trusted data and human review.
- Treat every fluent generated response as computationally correct or use the same model unchanged for unrelated task types.
- Compare use-case requirements, deterministic baselines, quantitative error rates, grounded outputs and exception handling.
- Trace shared feature pipelines, benchmark provenance, model supplier relationships, correlated misses and failover tests.
- Assess individual and group impact before deployment, require proportional human intervention and record appeal and remediation processes.
Explanation
Correct Answers
A:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
C:The reason it is necessary is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision. The acceptable verified outcome is: Selected components meet task-specific accuracy, controllability and auditability tests on representative work.
Incorrect Answers
B:For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
D:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. For this situation: A call center proposes a classification model to draft empathetic personalized customer letters and has not checked task suitability.
E:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. The decisive principle here is: Generative fluency and predictive statistical accuracy address different tasks; auditors must test fitness for a defined business decision.
