Isaca AAIA Practice Test 21
Question 200
An independent AI auditor reviews this case: An insurance algorithm passes aggregate accuracy but underprices claims for a vulnerable demographic subgroup. Which THREE elements together provide a defensible AI audit conclusion: the corrective control, the evidence to test it, and a verified outcome? Select THREE answers.
- Assess subgroup outcomes, proxy variables, relevant legitimate factors and mitigations with lawful context.
- Claim discrimination is impossible because the feature table no longer contains a protected-class column.
- Fairness monitoring detects problematic disparities and remediation does not conceal residual performance impacts.
- Document inherent versus residual risk using tested control effectiveness and authorized business acceptance.
- Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
- Compare justified subgroup metrics, candidate proxies, sampling context, error costs and approved fairness criteria.
Explanation
Correct Answers
A:The reason it is necessary is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity. For this situation: An insurance algorithm passes aggregate accuracy but underprices claims for a vulnerable demographic subgroup.
C:The reason it is necessary is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity. Independent audit evidence includes: Compare justified subgroup metrics, candidate proxies, sampling context, error costs and approved fairness criteria.
F:The reason it is necessary is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity. The corrective action is: Assess subgroup outcomes, proxy variables, relevant legitimate factors and mitigations with lawful context.
Incorrect Answers
B:For this situation: An insurance algorithm passes aggregate accuracy but underprices claims for a vulnerable demographic subgroup.
D:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. For this situation: An insurance algorithm passes aggregate accuracy but underprices claims for a vulnerable demographic subgroup.
E:This option concerns Adversarial risk and AI misuse scenarios, which is a related but different AI governance concern. The decisive principle here is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity.
Question 201
A governance and assurance review uncovers the following: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Assess subgroup outcomes, proxy variables, relevant legitimate factors and mitigations with lawful context.
- Track leading behavior, data-quality and control indicators with thresholds linked to risk appetite.
- Conduct a documented impact assessment of purpose, necessity, proportionality, alternatives and safeguards before deployment.
- Claim discrimination is impossible because the feature table no longer contains a protected-class column.
Explanation
Correct Answer
A:The reason it is necessary is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity. In this case: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
Incorrect Answers
B:This option concerns AI risk monitoring through leading KRIs, which is a related but different AI governance concern. In this case: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
C:This option concerns Privacy impact assessment for biometric AI, which is a related but different AI governance concern. In this case: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
D:This attractive shortcut does not resolve the Fairness assessment for proxy discrimination weakness: Claim discrimination is impossible because the feature table no longer contains a protected-class column. In this case: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
Question 202
A risk-based audit engagement examines this operating condition: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- AI risk assessment must consider intended context and foreseeable misuse, not only traditional IT incidents.
- Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity.
- Claim discrimination is impossible because the feature table no longer contains a protected-class column.
- High-impact monitoring of people can cause harm even when encryption and access controls are sound.
Explanation
Correct Answer
B:The acceptable verified outcome is: Fairness monitoring detects problematic disparities and remediation does not conceal residual performance impacts. In this case: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
Incorrect Answers
A:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. The decisive principle here is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity.
C:This attractive shortcut does not resolve the Fairness assessment for proxy discrimination weakness: Claim discrimination is impossible because the feature table no longer contains a protected-class column. For this situation: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
D:This option concerns Privacy impact assessment for biometric AI, which is a related but different AI governance concern. For this situation: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
Question 203
An audit committee asks for a defensible assessment of this situation: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect initial impact, treatment choice, operating tests, risk acceptance authority and current residual metrics.
- Claim discrimination is impossible because the feature table no longer contains a protected-class column.
- Inspect monitoring design, owners, thresholds, near-misses, drift and documented actions for breaches.
- Compare justified subgroup metrics, candidate proxies, sampling context, error costs and approved fairness criteria.
Explanation
Correct Answer
D:The acceptable verified outcome is: Fairness monitoring detects problematic disparities and remediation does not conceal residual performance impacts.
Incorrect Answers
A:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. The decisive principle here is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity.
B:For this situation: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
C:This option concerns AI risk monitoring through leading KRIs, which is a related but different AI governance concern. For this situation: A recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps.
Question 204
During an AI controls assessment, the team learns that a recruitment classifier removes sex from features but resumes reveal gender through proxy terms and career gaps. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Use case proceeds only with documented lawful purpose and proportionate risk treatment or is stopped.
- Claim discrimination is impossible because the feature table no longer contains a protected-class column.
- Fairness monitoring detects problematic disparities and remediation does not conceal residual performance impacts.
- Critical decisions continue or fail safely during vendor disruption with documented recovery alternatives.
Explanation
Correct Answer
C:This is an observable validation result for Fairness assessment for proxy discrimination, not just management assurance: Fairness monitoring detects problematic disparities and remediation does not conceal residual performance impacts.
Incorrect Answers
A:This option concerns Privacy impact assessment for biometric AI, which is a related but different AI governance concern. The audit must corroborate: Compare justified subgroup metrics, candidate proxies, sampling context, error costs and approved fairness criteria.
B:It misses the key control principle: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity.
D:This option concerns Model concentration and systemic vendor dependency, which is a related but different AI governance concern. The decisive principle here is: Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity.
Question 205
An independent AI auditor reviews this case: A fraud model trained before a payment-method shift suddenly flags most legitimate mobile-wallet purchases. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
- Document inherent versus residual risk using tested control effectiveness and authorized business acceptance.
- Assess subgroup outcomes, proxy variables, relevant legitimate factors and mitigations with lawful context.
- Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
Explanation
Correct Answer
D:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. This is the proportionate control response to Concept drift and changing customer behavior: Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
Incorrect Answers
A:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change.
B:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. It does not establish control effectiveness for Concept drift and changing customer behavior.
C:This option concerns Fairness assessment for proxy discrimination, which is a related but different AI governance concern. It does not establish control effectiveness for Concept drift and changing customer behavior.
Question 206
A governance and assurance review uncovers the following: A fraud model trained before a payment-method shift suddenly flags most legitimate mobile-wallet purchases. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
- Model suitability depends on performance, explainability and consequences of errors; a marginal metric gain does not remove oversight obligations.
- Mitigation exists only when control performance reduces measured exposure; a planned action is not risk reduction.
- Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
Explanation
Correct Answer
D:The corrective action is: Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions. This is the applicable AI audit principle for Concept drift and changing customer behavior: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
Incorrect Answers
A:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. For this situation: A fraud model trained before a payment-method shift suddenly flags most legitimate mobile-wallet purchases.
B:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. It does not establish control effectiveness for Concept drift and changing customer behavior.
C:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. The decisive principle here is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
Question 207
A risk-based audit engagement examines this operating condition: A fraud model trained before a payment-method shift suddenly flags most legitimate mobile-wallet purchases. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect initial impact, treatment choice, operating tests, risk acceptance authority and current residual metrics.
- Review dependency map, provider incidents, contract exit clauses, portability tests and fallback workflows.
- Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
Explanation
Correct Answer
C:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. The corrective action is: Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
Incorrect Answers
A:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. For this situation: A fraud model trained before a payment-method shift suddenly flags most legitimate mobile-wallet purchases.
B:This option concerns Model concentration and systemic vendor dependency, which is a related but different AI governance concern. It does not establish control effectiveness for Concept drift and changing customer behavior.
D:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. The auditor should instead examine: Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
Question 208
An audit committee asks for a defensible assessment of this situation: A fraud model trained before a payment-method shift suddenly flags most legitimate mobile-wallet purchases. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Material plausible failure scenarios have accountable controls, measurements and response plans.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
- Critical decisions continue or fail safely during vendor disruption with documented recovery alternatives.
- Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
Explanation
Correct Answer
D:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. This is an observable validation result for Concept drift and changing customer behavior, not just management assurance: Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
Incorrect Answers
A:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. It does not establish control effectiveness for Concept drift and changing customer behavior.
B:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
C:This option concerns Model concentration and systemic vendor dependency, which is a related but different AI governance concern. The decisive principle here is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
Question 209
During an AI controls assessment, the team learns that a sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Document inherent versus residual risk using tested control effectiveness and authorized business acceptance.
- Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
- Require decision-level explanations proportionate to the use case, validated predictive benefit, documented limitations and a meaningful appeal path.
Explanation
Correct Answer
B:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. The acceptable verified outcome is: Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
Incorrect Answers
A:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. In this case: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
C:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. In this case: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
D:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. The decisive principle here is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
Question 210
An independent AI auditor reviews this case: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Assess subgroup outcomes, proxy variables, relevant legitimate factors and mitigations with lawful context.
- Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
- Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
- Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
Explanation
Correct Answers
B:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
D:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. In this case: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
Incorrect Answers
A:This option concerns Fairness assessment for proxy discrimination, which is a related but different AI governance concern. The decisive principle here is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
C:This option concerns Adversarial risk and AI misuse scenarios, which is a related but different AI governance concern. It does not establish control effectiveness for Concept drift and changing customer behavior.
E:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
