Isaca AAIA Practice Test 22
Question 210
An independent AI auditor reviews this case: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Assess subgroup outcomes, proxy variables, relevant legitimate factors and mitigations with lawful context.
- Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
- Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
- Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
Explanation
Correct Answers
B:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
D:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. In this case: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
Incorrect Answers
A:This option concerns Fairness assessment for proxy discrimination, which is a related but different AI governance concern. The decisive principle here is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
C:This option concerns Adversarial risk and AI misuse scenarios, which is a related but different AI governance concern. It does not establish control effectiveness for Concept drift and changing customer behavior.
E:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
Question 211
A governance and assurance review uncovers the following: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
- Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
- Inspect use-case criticality, model comparison, reason-code fidelity, feature lineage, adverse-decision samples and appeal test results.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
Explanation
Correct Answer
A:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. This evidence directly tests the asserted control for Concept drift and changing customer behavior: Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
Incorrect Answers
B:This option concerns Adversarial risk and AI misuse scenarios, which is a related but different AI governance concern. For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
C:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
D:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. Effective closure requires: Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
Question 212
A risk-based audit engagement examines this operating condition: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
- Residual risk is accepted within appetite or escalated with validated control improvements.
- Material plausible failure scenarios have accountable controls, measurements and response plans.
- Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
Explanation
Correct Answer
D:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. Independent audit evidence includes: Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
Incorrect Answers
A:For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
B:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. For this situation: A sales forecast loses accuracy after rapid inflation but the model owner watches only uptime statistics.
C:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. The decisive principle here is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
Question 213
An audit committee asks for a defensible assessment of this situation: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
- Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
- Develop use-case-specific scenarios covering people, business process, privacy, safety, security and model behavior.
- Document inherent versus residual risk using tested control effectiveness and authorized business acceptance.
Explanation
Correct Answer
B:The reason it is necessary is: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged. In this case: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
Incorrect Answers
A:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. In this case: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
C:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. In this case: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
D:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. In this case: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
Question 214
During an AI controls assessment, the team learns that a disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
- Mitigation exists only when control performance reduces measured exposure; a planned action is not risk reduction.
- Removing an explicit sensitive field does not guarantee fairness if correlated inputs and outcomes retain inequity.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
Explanation
Correct Answer
A:The corrective action is: Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions. The acceptable verified outcome is: Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
Incorrect Answers
B:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. For this situation: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
C:This option concerns Fairness assessment for proxy discrimination, which is a related but different AI governance concern. In this case: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
D:This attractive shortcut does not resolve the Concept drift and changing customer behavior weakness: Assume a model remains accurate forever because its API responds successfully and weights never change. For this situation: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
Question 215
An independent AI auditor reviews this case: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
- Review use-case map, stakeholders, failure modes, risk register, impact assumptions and challenge workshops.
- Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
- Compare justified subgroup metrics, candidate proxies, sampling context, error costs and approved fairness criteria.
Explanation
Correct Answer
C:The corrective action is: Define drift indicators and use-case-specific revalidation triggers with approved mitigation decisions.
Incorrect Answers
A:For this situation: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
B:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. For this situation: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
D:This option concerns Fairness assessment for proxy discrimination, which is a related but different AI governance concern. For this situation: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
Question 216
A governance and assurance review uncovers the following: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Sampled denials have faithful understandable reasons, reliable escalation and documented accuracy versus simpler alternatives.
- Material plausible failure scenarios have accountable controls, measurements and response plans.
- Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
- Assume a model remains accurate forever because its API responds successfully and weights never change.
Explanation
Correct Answer
C:This is an observable validation result for Concept drift and changing customer behavior, not just management assurance: Drift alerts trigger timely review and validated model changes without uncontrolled retraining.
Incorrect Answers
A:This option concerns Explainable credit-model selection for adverse decisions, which is a related but different AI governance concern. In this case: A disease-risk predictor calibrated on adults is repurposed for children without recalibration or domain testing.
B:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. The audit must corroborate: Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
D:It misses the key control principle: Data and concept distributions evolve; model performance can decay even when code and infrastructure remain unchanged.
Question 217
A risk-based audit engagement examines this operating condition: A public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Develop use-case-specific scenarios covering people, business process, privacy, safety, security and model behavior.
- Quantify common-provider concentration and design alternative operations, exit and service continuity according to risk.
- Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference.
- Threat-model model-specific abuse paths, adversary access, safeguards and residual exposure.
Explanation
Correct Answer
D:The acceptable verified outcome is: Risk register and tests cover plausible exploitation paths, and high-impact gaps are prioritized. This is the proportionate control response to Adversarial risk and AI misuse scenarios: Threat-model model-specific abuse paths, adversary access, safeguards and residual exposure.
Incorrect Answers
A:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. It does not establish control effectiveness for Adversarial risk and AI misuse scenarios.
B:This option concerns Model concentration and systemic vendor dependency, which is a related but different AI governance concern. It does not establish control effectiveness for Adversarial risk and AI misuse scenarios.
C:This attractive shortcut does not resolve the Adversarial risk and AI misuse scenarios weakness: Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference.
Question 218
An audit committee asks for a defensible assessment of this situation: A public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- AI risk assessment must consider intended context and foreseeable misuse, not only traditional IT incidents.
- Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference.
- Mitigation exists only when control performance reduces measured exposure; a planned action is not risk reduction.
- AI attack surfaces include data, models, prompts, output use and agent capabilities beyond traditional network endpoints.
Explanation
Correct Answer
D:The acceptable verified outcome is: Risk register and tests cover plausible exploitation paths, and high-impact gaps are prioritized. This is the applicable AI audit principle for Adversarial risk and AI misuse scenarios: AI attack surfaces include data, models, prompts, output use and agent capabilities beyond traditional network endpoints.
Incorrect Answers
A:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. For this situation: A public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference.
B:This attractive shortcut does not resolve the Adversarial risk and AI misuse scenarios weakness: Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference. For this situation: A public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference.
C:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. It does not establish control effectiveness for Adversarial risk and AI misuse scenarios.
Question 219
During an AI controls assessment, the team learns that a public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Review use-case map, stakeholders, failure modes, risk register, impact assumptions and challenge workshops.
- Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference.
- Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
- Inspect temporal validation, segment error, data drift, concept labels, release dates and retraining criteria.
Explanation
Correct Answer
C:The acceptable verified outcome is: Risk register and tests cover plausible exploitation paths, and high-impact gaps are prioritized. This evidence directly tests the asserted control for Adversarial risk and AI misuse scenarios: Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
Incorrect Answers
A:This option concerns AI use-case risk scenario identification, which is a related but different AI governance concern. The decisive principle here is: AI attack surfaces include data, models, prompts, output use and agent capabilities beyond traditional network endpoints.
B:This attractive shortcut does not resolve the Adversarial risk and AI misuse scenarios weakness: Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference. It misses the key control principle: AI attack surfaces include data, models, prompts, output use and agent capabilities beyond traditional network endpoints.
D:This option concerns Concept drift and changing customer behavior, which is a related but different AI governance concern. It does not establish control effectiveness for Adversarial risk and AI misuse scenarios.
Question 220
An independent AI auditor reviews this case: A public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
- Track leading behavior, data-quality and control indicators with thresholds linked to risk appetite.
- Inspect initial impact, treatment choice, operating tests, risk acceptance authority and current residual metrics.
- Threat-model model-specific abuse paths, adversary access, safeguards and residual exposure.
- Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference.
Explanation
Correct Answers
A:The acceptable verified outcome is: Risk register and tests cover plausible exploitation paths, and high-impact gaps are prioritized. For this situation: A public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference.
D:The acceptable verified outcome is: Risk register and tests cover plausible exploitation paths, and high-impact gaps are prioritized. This response fits the risk associated with Adversarial risk and AI misuse scenarios: Threat-model model-specific abuse paths, adversary access, safeguards and residual exposure.
Incorrect Answers
B:This option concerns AI risk monitoring through leading KRIs, which is a related but different AI governance concern. It does not establish control effectiveness for Adversarial risk and AI misuse scenarios.
C:This option concerns Residual AI risk acceptance and mitigation efficacy, which is a related but different AI governance concern. For this situation: A public model endpoint allows repeated probing but the threat register ignores model extraction and membership inference.
E:This attractive shortcut does not resolve the Adversarial risk and AI misuse scenarios weakness: Assume ordinary perimeter firewalls address all adversarial manipulation of training and inference. The auditor should instead examine: Inspect architecture data flows, attacker capabilities, attack trees, abuse test coverage and limiting controls.
