Isaca AAIA Practice Test 105
Question 1040
An independent AI auditor reviews this case: A team draws a judgmental sample of prompts and presents it as an unbiased estimate for the entire population. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Match attribute, variables and judgmental methods to the test purpose with transparent population and exception logic.
- Project an auditor-selected convenience sample to the full deployment as if it were statistically random.
- Preserve professional objectivity, disclose conflicts and independently validate AI-assisted audit decisions.
- Inspect population completeness, test objective, sampling design, tolerable error and audit projection rules.
- Inspect service organization reports, contract rights, complementary controls, model monitoring and exception records.
Explanation
Correct Answers
A:The acceptable verified outcome is: Reported conclusions state what the sample supports and do not overgeneralize judgmental observations. This response fits the risk associated with Attribute and variable sampling for AI controls: Match attribute, variables and judgmental methods to the test purpose with transparent population and exception logic.
D:The acceptable verified outcome is: Reported conclusions state what the sample supports and do not overgeneralize judgmental observations. For this situation: A team draws a judgmental sample of prompts and presents it as an unbiased estimate for the entire population.
Incorrect Answers
B:For this situation: A team draws a judgmental sample of prompts and presents it as an unbiased estimate for the entire population.
C:This option concerns Auditor independence with AI-enabled workpapers, which is a related but different AI governance concern. The decisive principle here is: Sampling claims require valid method, controls over selection and a clear interpretation of exceptions.
E:This option concerns Third-party AI assurance boundary and audit rights, which is a related but different AI governance concern. The outcome to verify is: Reported conclusions state what the sample supports and do not overgeneralize judgmental observations.
Question 1041
A governance and assurance review uncovers the following: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Project an auditor-selected convenience sample to the full deployment as if it were statistically random.
- Sample across relevant periods, model versions and operational shifts and reconstruct point-in-time inputs.
- Match attribute, variables and judgmental methods to the test purpose with transparent population and exception logic.
- Combine risk-weighted stratification and justified random sampling across consequential outcomes and populations.
Explanation
Correct Answer
C:The acceptable verified outcome is: Reported conclusions state what the sample supports and do not overgeneralize judgmental observations. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
Incorrect Answers
A:This attractive shortcut does not resolve the Attribute and variable sampling for AI controls weakness: Project an auditor-selected convenience sample to the full deployment as if it were statistically random. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
B:This option concerns Time-based audit testing for drifting AI systems, which is a related but different AI governance concern. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
D:This option concerns Stratified sampling for rare AI harms, which is a related but different AI governance concern. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
Question 1042
A risk-based audit engagement examines this operating condition: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Sampling claims require valid method, controls over selection and a clear interpretation of exceptions.
- Project an auditor-selected convenience sample to the full deployment as if it were statistically random.
- Audit independence extends to automated methods and relationships that could bias scope or findings.
- Findings must be based on traceable and testable criteria rather than a reviewer's undefined preference.
Explanation
Correct Answer
A:The acceptable verified outcome is: Reported conclusions state what the sample supports and do not overgeneralize judgmental observations. This is the applicable AI audit principle for Attribute and variable sampling for AI controls: Sampling claims require valid method, controls over selection and a clear interpretation of exceptions.
Incorrect Answers
B:This attractive shortcut does not resolve the Attribute and variable sampling for AI controls weakness: Project an auditor-selected convenience sample to the full deployment as if it were statistically random. For this situation: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
C:This option concerns Auditor independence with AI-enabled workpapers, which is a related but different AI governance concern. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
D:This option concerns AI audit criteria and control objectives, which is a related but different AI governance concern. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
Question 1043
An audit committee asks for a defensible assessment of this situation: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Project an auditor-selected convenience sample to the full deployment as if it were statistically random.
- Inspect source test inputs, environment version, expected result, output logs and exception handling.
- Reconcile AI inventory, data flows, business ownership, complaints, material incidents and risk appetite to audit plan.
- Inspect population completeness, test objective, sampling design, tolerable error and audit projection rules.
Explanation
Correct Answer
D:In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
Incorrect Answers
A:For this situation: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
B:This option concerns Reperformance and adversarial audit test cases, which is a related but different AI governance concern. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
C:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
Question 1044
During an AI controls assessment, the team learns that an AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Reported conclusions state what the sample supports and do not overgeneralize judgmental observations.
- The plan covers material high-risk deployments and states documented scope boundaries and assurance limitations.
- Independent repeat tests reproduce control behavior and deviations generate substantiated findings.
- Project an auditor-selected convenience sample to the full deployment as if it were statistically random.
Explanation
Correct Answer
A:The reason it is necessary is: Sampling claims require valid method, controls over selection and a clear interpretation of exceptions. In this case: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
Incorrect Answers
B:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. For this situation: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
C:This option concerns Reperformance and adversarial audit test cases, which is a related but different AI governance concern. For this situation: An AI control test uses a zero-error acceptance rule without defining tolerance or sampling confidence.
D:It misses the key control principle: Sampling claims require valid method, controls over selection and a clear interpretation of exceptions.
Question 1045
An independent AI auditor reviews this case: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
- Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible.
- Reperform representative positive and negative controls using independently selected inputs and authorized challenge procedures.
- Match attribute, variables and judgmental methods to the test purpose with transparent population and exception logic.
Explanation
Correct Answer
C:The reason it is necessary is: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions. In this case: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests.
Incorrect Answers
A:This option concerns Audit data completeness and reconciliation, which is a related but different AI governance concern. It does not establish control effectiveness for Reperformance and adversarial audit test cases.
B:This attractive shortcut does not resolve the Reperformance and adversarial audit test cases weakness: Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible.
D:This option concerns Attribute and variable sampling for AI controls, which is a related but different AI governance concern. It does not establish control effectiveness for Reperformance and adversarial audit test cases.
Question 1046
A governance and assurance review uncovers the following: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions.
- Audit scope must follow enterprise AI exposure, not only visible software projects or infrastructure boundaries.
- Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible.
- Sampling claims require valid method, controls over selection and a clear interpretation of exceptions.
Explanation
Correct Answer
A:Independent audit evidence includes: Inspect source test inputs, environment version, expected result, output logs and exception handling. In this case: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests.
Incorrect Answers
B:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. It does not establish control effectiveness for Reperformance and adversarial audit test cases.
C:This attractive shortcut does not resolve the Reperformance and adversarial audit test cases weakness: Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible. For this situation: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests.
D:This option concerns Attribute and variable sampling for AI controls, which is a related but different AI governance concern. For this situation: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests.
Question 1047
A risk-based audit engagement examines this operating condition: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect source test inputs, environment version, expected result, output logs and exception handling.
- Inspect population completeness, test objective, sampling design, tolerable error and audit projection rules.
- Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible.
- Inspect audit program, control objectives, regulatory applicability, measurements and management agreement on criteria.
Explanation
Correct Answer
A:The reason it is necessary is: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions. This evidence directly tests the asserted control for Reperformance and adversarial audit test cases: Inspect source test inputs, environment version, expected result, output logs and exception handling.
Incorrect Answers
B:This option concerns Attribute and variable sampling for AI controls, which is a related but different AI governance concern. The decisive principle here is: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions.
C:This attractive shortcut does not resolve the Reperformance and adversarial audit test cases weakness: Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible. It misses the key control principle: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions.
D:This option concerns AI audit criteria and control objectives, which is a related but different AI governance concern. It does not establish control effectiveness for Reperformance and adversarial audit test cases.
Question 1048
An audit committee asks for a defensible assessment of this situation: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Independent repeat tests reproduce control behavior and deviations generate substantiated findings.
- The plan covers material high-risk deployments and states documented scope boundaries and assurance limitations.
- Workpapers map each issue to an explicit criterion, condition, cause, consequence and evidence.
- Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible.
Explanation
Correct Answer
A:The reason it is necessary is: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions. This is an observable validation result for Reperformance and adversarial audit test cases, not just management assurance: Independent repeat tests reproduce control behavior and deviations generate substantiated findings.
Incorrect Answers
B:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. For this situation: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests.
C:This option concerns AI audit criteria and control objectives, which is a related but different AI governance concern. For this situation: An audit team accepts a model owner screenshot showing successful safety filtering rather than testing forbidden requests.
D:This attractive shortcut does not resolve the Reperformance and adversarial audit test cases weakness: Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible. The auditor should instead examine: Inspect source test inputs, environment version, expected result, output logs and exception handling.
Question 1049
During an AI controls assessment, the team learns that a bank gives auditors only handpicked fair-decision examples and refuses independently selected records. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Reperform representative positive and negative controls using independently selected inputs and authorized challenge procedures.
- Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible.
- Reconcile source populations, failed records, joins and transformations before interpreting AI audit analytics.
- Define the AI audit universe and scope using use-case impact, third parties, control reliance and key risk scenarios.
Explanation
Correct Answer
A:The reason it is necessary is: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions. In this case: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records.
Incorrect Answers
B:This attractive shortcut does not resolve the Reperformance and adversarial audit test cases weakness: Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible. In this case: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records.
C:This option concerns Audit data completeness and reconciliation, which is a related but different AI governance concern. In this case: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records.
D:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. In this case: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records.
Question 1050
An independent AI auditor reviews this case: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect source test inputs, environment version, expected result, output logs and exception handling.
- Inspect version timelines, drift monitors, incident dates, feature snapshots and time-based sampling criteria.
- Define the AI audit universe and scope using use-case impact, third parties, control reliance and key risk scenarios.
- Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible.
- Reperform representative positive and negative controls using independently selected inputs and authorized challenge procedures.
Explanation
Correct Answers
A:The reason it is necessary is: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions. The corrective action is: Reperform representative positive and negative controls using independently selected inputs and authorized challenge procedures.
E:The reason it is necessary is: Control operation is better evidenced by observed outcomes and reproducible test steps than management assertions. For this situation: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records.
Incorrect Answers
B:This option concerns Time-based audit testing for drifting AI systems, which is a related but different AI governance concern. It does not establish control effectiveness for Reperformance and adversarial audit test cases.
C:This option concerns Risk-based AI audit universe and scope, which is a related but different AI governance concern. For this situation: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records.
D:This attractive shortcut does not resolve the Reperformance and adversarial audit test cases weakness: Use vendor screenshots or scripted demonstrations as substitutes for independent evidence when testing is feasible. For this situation: A bank gives auditors only handpicked fair-decision examples and refuses independently selected records.