Isaca AAIA Practice Test 5
Question 40
An independent AI auditor reviews this case: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Evaluate training distribution, drift, thresholds, false-positive harms and human triage before using unsupervised scores for consequential actions.
- Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- Sample user interface, override permissions, time pressure, rationale records and actual reviewer interventions.
Explanation
Correct Answers
B:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. For this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
C:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. This evidence directly tests the asserted control for Retrieval-augmented answer grounding: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Incorrect Answers
A:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. The decisive principle here is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
D:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
E:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. For this situation: A legal helpdesk bot cites policy documents it did not retrieve and invents paragraph numbers for employee benefits questions.
Question 41
A governance and assurance review uncovers the following: An enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate shared data, dependencies and failure modes of model components, rather than counting models as independent assurance.
- Maintain a current AI system inventory covering owned, procured and embedded capabilities, data flows, owners and criticality.
- Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
Explanation
Correct Answer
C:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. Independent audit evidence includes: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Incorrect Answers
A:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. The audit must corroborate: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
B:This option concerns Enterprise AI inventory and shadow-AI discovery, which is a related but different AI governance concern. It does not establish control effectiveness for Retrieval-augmented answer grounding.
D:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. Effective closure requires: A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Question 42
A risk-based audit engagement examines this operating condition: An enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- An anomaly is a deviation from a learned baseline, not an independently verified fraud or misconduct conclusion.
- Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
- Governance cannot assess or monitor AI risk if material use cases and external processors are unknown.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
Explanation
Correct Answer
B:The corrective action is: Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference. Independent audit evidence includes: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Incorrect Answers
A:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. The audit must corroborate: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
C:This option concerns Enterprise AI inventory and shadow-AI discovery, which is a related but different AI governance concern. The decisive principle here is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
D:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. For this situation: An enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules.
Question 43
An audit committee asks for a defensible assessment of this situation: An enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Examine decision map, error costs, affected populations, human override authority, complaints and impact assessment.
- Sample user interface, override permissions, time pressure, rationale records and actual reviewer interventions.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Explanation
Correct Answer
D:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. The acceptable verified outcome is: A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Incorrect Answers
A:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. It does not establish control effectiveness for Retrieval-augmented answer grounding.
B:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. The decisive principle here is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
C:It misses the key control principle: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
Question 44
During an AI controls assessment, the team learns that an enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Material AI uses are inventoried, risk-tiered and linked to approved business/data owners before production use.
- Independent challenge tests detect correlated failure and evidence supports any reliability uplift claimed for the ensemble.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Explanation
Correct Answer
D:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. This is an observable validation result for Retrieval-augmented answer grounding, not just management assurance: A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Incorrect Answers
A:This option concerns Enterprise AI inventory and shadow-AI discovery, which is a related but different AI governance concern. For this situation: An enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules.
B:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. For this situation: An enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules.
C:For this situation: An enterprise knowledge assistant answers from a stale index despite a current policy revision that changed entitlement rules.
Question 45
An independent AI auditor reviews this case: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- Evaluate shared data, dependencies and failure modes of model components, rather than counting models as independent assurance.
- Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
- Assess individual and group impact before deployment, require proportional human intervention and record appeal and remediation processes.
Explanation
Correct Answer
C:The reason it is necessary is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims. In this case: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
Incorrect Answers
A:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. In this case: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
B:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. In this case: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
D:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. The decisive principle here is: Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
Question 46
A governance and assurance review uncovers the following: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- A human rubber stamp without information or authority does not provide effective human oversight.
- Retrieval can reduce unsupported answers only if accessible current material is relevant and citations actually support claims.
- A useful aggregate prediction may still cause disproportionate consequences for affected people and demands context-specific governance.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
Explanation
Correct Answer
B:The corrective action is: Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference. In this case: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
Incorrect Answers
A:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. In this case: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
C:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. For this situation: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
D:This attractive shortcut does not resolve the Retrieval-augmented answer grounding weakness: Assume attaching any document URL to an answer proves the answer was grounded in that document. For this situation: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
Question 47
A risk-based audit engagement examines this operating condition: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Trace shared feature pipelines, benchmark provenance, model supplier relationships, correlated misses and failover tests.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
- Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
Explanation
Correct Answer
C:The corrective action is: Require traceable retrieved-source references, corpus version controls and answer checks that distinguish source evidence from inference.
Incorrect Answers
A:This option concerns Multi-model ensemble risk and failure correlation, which is a related but different AI governance concern. For this situation: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
B:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. In this case: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
D:For this situation: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
Question 48
An audit committee asks for a defensible assessment of this situation: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Flagged cases receive contextual review with measured precision, recall proxies and acceptable false-intervention rates.
- Adverse outcomes, challenge rights and corrections are monitored with approved accountable decision owners.
- Assume attaching any document URL to an answer proves the answer was grounded in that document.
- A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Explanation
Correct Answer
D:This is an observable validation result for Retrieval-augmented answer grounding, not just management assurance: A stratified answer test shows supported citations, correct version use and safe abstention when evidence is absent.
Incorrect Answers
A:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. For this situation: A procurement chatbot retrieves text from a vendor proposal but presents the vendor claim as verified internal policy.
B:This option concerns Automated decision impact on individuals, which is a related but different AI governance concern. The audit must corroborate: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
C:The auditor should instead examine: Inspect retrieval traces, passage IDs, document dates, authorization filters, answer citation fidelity and no-answer cases.
Question 49
During an AI controls assessment, the team learns that a hospital proposes an AI triage score that reallocates appointments without assessing delays for patients with disabilities. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Design usable human review with understandable context, authority to intervene, escalation and traceable decisions.
- Assess individual and group impact before deployment, require proportional human intervention and record appeal and remediation processes.
- Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
- Equate higher average accuracy with evidence that no person can be harmed by an automated decision.
Explanation
Correct Answer
B:The reason it is necessary is: A useful aggregate prediction may still cause disproportionate consequences for affected people and demands context-specific governance. In this case: A hospital proposes an AI triage score that reallocates appointments without assessing delays for patients with disabilities.
Incorrect Answers
A:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. It does not establish control effectiveness for Automated decision impact on individuals.
C:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. It does not establish control effectiveness for Automated decision impact on individuals.
D:This attractive shortcut does not resolve the Automated decision impact on individuals weakness: Equate higher average accuracy with evidence that no person can be harmed by an automated decision.
Question 50
An independent AI auditor reviews this case: A hospital proposes an AI triage score that reallocates appointments without assessing delays for patients with disabilities. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Assess individual and group impact before deployment, require proportional human intervention and record appeal and remediation processes.
- Equate higher average accuracy with evidence that no person can be harmed by an automated decision.
- Design usable human review with understandable context, authority to intervene, escalation and traceable decisions.
- Review cluster validation, anomaly score distributions, seasonal control populations, downstream actions and analyst adjudications.
- Examine decision map, error costs, affected populations, human override authority, complaints and impact assessment.
Explanation
Correct Answers
A:The reason it is necessary is: A useful aggregate prediction may still cause disproportionate consequences for affected people and demands context-specific governance. For this situation: A hospital proposes an AI triage score that reallocates appointments without assessing delays for patients with disabilities.
E:The reason it is necessary is: A useful aggregate prediction may still cause disproportionate consequences for affected people and demands context-specific governance. This evidence directly tests the asserted control for Automated decision impact on individuals: Examine decision map, error costs, affected populations, human override authority, complaints and impact assessment.
Incorrect Answers
B:This attractive shortcut does not resolve the Automated decision impact on individuals weakness: Equate higher average accuracy with evidence that no person can be harmed by an automated decision. For this situation: A hospital proposes an AI triage score that reallocates appointments without assessing delays for patients with disabilities.
C:This option concerns Human oversight design for high-impact AI, which is a related but different AI governance concern. For this situation: A hospital proposes an AI triage score that reallocates appointments without assessing delays for patients with disabilities.
D:This option concerns Unsupervised anomaly detection and alert governance, which is a related but different AI governance concern. It does not establish control effectiveness for Automated decision impact on individuals.
