Isaca AAIA Practice Test 111
Question 1100
An independent AI auditor reviews this case: An incident team cannot identify which deployed prompt and model version caused a confidential citation leak. Which THREE elements together provide a defensible AI audit conclusion: the corrective control, the evidence to test it, and a verified outcome? Select THREE answers.
- Capture model and prompt versions, retrieval sources, seeds where relevant and external state for reproducible testing.
- Review engagement conflicts, tool ownership, reviewer signoff, challenge evidence and decision trail.
- Inspect deployment identifiers, prompt hashes, retrieval snapshots, tool trace IDs and evaluation settings.
- Auditors can reproduce or defensibly explain a decision with known context and documented stochastic limits.
- Plan third-party evidence access, complementary user controls, limitation statements and risk-based alternative procedures.
- Treat a single new model response as definitive evidence of the exact historical output under changed conditions.
Explanation
Correct Answers
A:The reason it is necessary is: Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound. For this situation: An incident team cannot identify which deployed prompt and model version caused a confidential citation leak.
C:The reason it is necessary is: Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound. This evidence directly tests the asserted control for Model version and retrieval evidence preservation: Inspect deployment identifiers, prompt hashes, retrieval snapshots, tool trace IDs and evaluation settings.
D:The reason it is necessary is: Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound. Independent audit evidence includes: Inspect deployment identifiers, prompt hashes, retrieval snapshots, tool trace IDs and evaluation settings.
Incorrect Answers
B:This option concerns Auditor independence with AI-enabled workpapers, which is a related but different AI governance concern. For this situation: An incident team cannot identify which deployed prompt and model version caused a confidential citation leak.
E:This option concerns Third-party AI assurance boundary and audit rights, which is a related but different AI governance concern. It does not establish control effectiveness for Model version and retrieval evidence preservation.
F:For this situation: An incident team cannot identify which deployed prompt and model version caused a confidential citation leak.
Question 1101
A governance and assurance review uncovers the following: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Plan third-party evidence access, complementary user controls, limitation statements and risk-based alternative procedures.
- Triangulate available independent evidence and document unresolved access limitations instead of overstating assurance.
- Treat a single new model response as definitive evidence of the exact historical output under changed conditions.
- Capture model and prompt versions, retrieval sources, seeds where relevant and external state for reproducible testing.
Explanation
Correct Answer
D:The reason it is necessary is: Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound. In this case: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
Incorrect Answers
A:This option concerns Third-party AI assurance boundary and audit rights, which is a related but different AI governance concern. The decisive principle here is: Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound.
B:This option concerns Evidence sufficiency when source model is inaccessible, which is a related but different AI governance concern. It does not establish control effectiveness for Model version and retrieval evidence preservation.
C:This attractive shortcut does not resolve the Model version and retrieval evidence preservation weakness: Treat a single new model response as definitive evidence of the exact historical output under changed conditions. In this case: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
Question 1102
A risk-based audit engagement examines this operating condition: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Evidence handling must protect confidentiality and integrity while maintaining the ability to demonstrate provenance.
- Outsourced AI decisions remain material to the customer organization even if source code is not accessible.
- Treat a single new model response as definitive evidence of the exact historical output under changed conditions.
- Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound.
Explanation
Correct Answer
D:The corrective action is: Capture model and prompt versions, retrieval sources, seeds where relevant and external state for reproducible testing. In this case: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
Incorrect Answers
A:This option concerns Chain of custody for sensitive AI audit extracts, which is a related but different AI governance concern. In this case: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
B:This option concerns Third-party AI assurance boundary and audit rights, which is a related but different AI governance concern. For this situation: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
C:This attractive shortcut does not resolve the Model version and retrieval evidence preservation weakness: Treat a single new model response as definitive evidence of the exact historical output under changed conditions. For this situation: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
Question 1103
An audit committee asks for a defensible assessment of this situation: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Reconcile AI inventory, data flows, business ownership, complaints, material incidents and risk appetite to audit plan.
- Inspect contractual audit rights, alternative records, sampled outputs, independent tests and unresolved exclusions.
- Inspect deployment identifiers, prompt hashes, retrieval snapshots, tool trace IDs and evaluation settings.
- Treat a single new model response as definitive evidence of the exact historical output under changed conditions.
Explanation
Correct Answer
C:The corrective action is: Capture model and prompt versions, retrieval sources, seeds where relevant and external state for reproducible testing.
Incorrect Answers
A:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. In this case: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
B:This option concerns Evidence sufficiency when source model is inaccessible, which is a related but different AI governance concern. For this situation: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
D:For this situation: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
Question 1104
During an AI controls assessment, the team learns that a generative model's outputs vary by temperature and external tool state but workpapers record only the user question. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Treat a single new model response as definitive evidence of the exact historical output under changed conditions.
- Final conclusions distinguish tested controls from unsupported claims and explain resulting scope restrictions.
- Workpapers map each issue to an explicit criterion, condition, cause, consequence and evidence.
- Auditors can reproduce or defensibly explain a decision with known context and documented stochastic limits.
Explanation
Correct Answer
D:This is an observable validation result for Model version and retrieval evidence preservation, not just management assurance: Auditors can reproduce or defensibly explain a decision with known context and documented stochastic limits.
Incorrect Answers
A:It misses the key control principle: Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound.
B:This option concerns Evidence sufficiency when source model is inaccessible, which is a related but different AI governance concern. The decisive principle here is: Reperformance of probabilistic or tool-using AI needs contextual version evidence and a documented uncertainty bound.
C:This option concerns AI audit criteria and control objectives, which is a related but different AI governance concern. In this case: A generative model's outputs vary by temperature and external tool state but workpapers record only the user question.
Question 1105
An independent AI auditor reviews this case: An AI fraud audit extracts a million scored transactions but forgets records that the model rejected before scoring. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Reperform representative positive and negative controls using independently selected inputs and authorized challenge procedures.
- Define the AI audit universe and scope using use-case impact, third parties, control reliance and key risk scenarios.
- Treat a large CSV row count as proof of complete AI decision coverage.
- Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
Explanation
Correct Answer
D:The reason it is necessary is: Incomplete populations and silent joins can bias control effectiveness conclusions. This is the proportionate control response to Audit data completeness and reconciliation: Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
Incorrect Answers
A:This option concerns Reperformance and adversarial audit test cases, which is a related but different AI governance concern. It does not establish control effectiveness for Audit data completeness and reconciliation.
B:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. It does not establish control effectiveness for Audit data completeness and reconciliation.
C:This attractive shortcut does not resolve the Audit data completeness and reconciliation weakness: Treat a large CSV row count as proof of complete AI decision coverage.
Question 1106
A governance and assurance review uncovers the following: An AI fraud audit extracts a million scored transactions but forgets records that the model rejected before scoring. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Findings must be based on traceable and testable criteria rather than a reviewer's undefined preference.
- Incomplete populations and silent joins can bias control effectiveness conclusions.
- Treat a large CSV row count as proof of complete AI decision coverage.
- Reliable audit analytics depend on traceable transformations, not merely attractive final charts.
Explanation
Correct Answer
B:The corrective action is: Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics. This is the applicable AI audit principle for Audit data completeness and reconciliation: Incomplete populations and silent joins can bias control effectiveness conclusions.
Incorrect Answers
A:This option concerns AI audit criteria and control objectives, which is a related but different AI governance concern. It does not establish control effectiveness for Audit data completeness and reconciliation.
C:This attractive shortcut does not resolve the Audit data completeness and reconciliation weakness: Treat a large CSV row count as proof of complete AI decision coverage. For this situation: An AI fraud audit extracts a million scored transactions but forgets records that the model rejected before scoring.
D:This option concerns Reproducible audit data transformations, which is a related but different AI governance concern. It does not establish control effectiveness for Audit data completeness and reconciliation.
Question 1107
A risk-based audit engagement examines this operating condition: An AI fraud audit extracts a million scored transactions but forgets records that the model rejected before scoring. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Treat a large CSV row count as proof of complete AI decision coverage.
- Inspect row counts, independent totals, missing keys, excluded statuses and transformation logs.
- Inspect audit program, control objectives, regulatory applicability, measurements and management agreement on criteria.
- Review challenge samples, model changes, error rates, reviewer overrides and data privacy restrictions.
Explanation
Correct Answer
B:The reason it is necessary is: Incomplete populations and silent joins can bias control effectiveness conclusions. The corrective action is: Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
Incorrect Answers
A:This attractive shortcut does not resolve the Audit data completeness and reconciliation weakness: Treat a large CSV row count as proof of complete AI decision coverage. The auditor should instead examine: Inspect row counts, independent totals, missing keys, excluded statuses and transformation logs.
C:This option concerns AI audit criteria and control objectives, which is a related but different AI governance concern. For this situation: An AI fraud audit extracts a million scored transactions but forgets records that the model rejected before scoring.
D:This option concerns Validation of AI-enabled audit analytics tools, which is a related but different AI governance concern. It does not establish control effectiveness for Audit data completeness and reconciliation.
Question 1108
An audit committee asks for a defensible assessment of this situation: An AI fraud audit extracts a million scored transactions but forgets records that the model rejected before scoring. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- End-to-end reconciliations explain missing and duplicated records before sampling and metrics.
- Treat a large CSV row count as proof of complete AI decision coverage.
- A second reviewer reproduces key findings from the recorded inputs and processing logic.
- An accountable auditor validates the final scope and retains evidence for AI-generated suggestions.
Explanation
Correct Answer
A:The reason it is necessary is: Incomplete populations and silent joins can bias control effectiveness conclusions. This is an observable validation result for Audit data completeness and reconciliation, not just management assurance: End-to-end reconciliations explain missing and duplicated records before sampling and metrics.
Incorrect Answers
B:This attractive shortcut does not resolve the Audit data completeness and reconciliation weakness: Treat a large CSV row count as proof of complete AI decision coverage. Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
C:This option concerns Reproducible audit data transformations, which is a related but different AI governance concern. For this situation: An AI fraud audit extracts a million scored transactions but forgets records that the model rejected before scoring.
D:This option concerns Using AI during audit engagement planning, which is a related but different AI governance concern. It does not establish control effectiveness for Audit data completeness and reconciliation.
Question 1109
During an AI controls assessment, the team learns that a model monitoring export reports only successful requests while the highest-risk failed decisions are absent. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Define objective risk-based audit criteria from approved organizational policies, relevant standards and applicable law.
- Treat a large CSV row count as proof of complete AI decision coverage.
- Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
- Define the AI audit universe and scope using use-case impact, third parties, control reliance and key risk scenarios.
Explanation
Correct Answer
C:The reason it is necessary is: Incomplete populations and silent joins can bias control effectiveness conclusions. The acceptable verified outcome is: End-to-end reconciliations explain missing and duplicated records before sampling and metrics.
Incorrect Answers
A:This option concerns AI audit criteria and control objectives, which is a related but different AI governance concern. The decisive principle here is: Incomplete populations and silent joins can bias control effectiveness conclusions.
B:This attractive shortcut does not resolve the Audit data completeness and reconciliation weakness: Treat a large CSV row count as proof of complete AI decision coverage. In this case: A model monitoring export reports only successful requests while the highest-risk failed decisions are absent.
D:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. The decisive principle here is: Incomplete populations and silent joins can bias control effectiveness conclusions.
Question 1110
An independent AI auditor reviews this case: A model monitoring export reports only successful requests while the highest-risk failed decisions are absent. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect row counts, independent totals, missing keys, excluded statuses and transformation logs.
- Validate audit-tool outputs against independent truth, monitor drift and use human judgment for material findings.
- Reconcile AI inventory, data flows, business ownership, complaints, material incidents and risk appetite to audit plan.
- Treat a large CSV row count as proof of complete AI decision coverage.
- Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
Explanation
Correct Answers
A:The reason it is necessary is: Incomplete populations and silent joins can bias control effectiveness conclusions. In this case: A model monitoring export reports only successful requests while the highest-risk failed decisions are absent.
E:The reason it is necessary is: Incomplete populations and silent joins can bias control effectiveness conclusions. For this situation: A model monitoring export reports only successful requests while the highest-risk failed decisions are absent.
Incorrect Answers
B:This option concerns Validation of AI-enabled audit analytics tools, which is a related but different AI governance concern. The decisive principle here is: Incomplete populations and silent joins can bias control effectiveness conclusions.
C:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. For this situation: A model monitoring export reports only successful requests while the highest-risk failed decisions are absent.
D:This attractive shortcut does not resolve the Audit data completeness and reconciliation weakness: Treat a large CSV row count as proof of complete AI decision coverage. For this situation: A model monitoring export reports only successful requests while the highest-risk failed decisions are absent.
