Isaca AAIA Practice Test 69
Question 680
An independent AI auditor reviews this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
- Design actionable challenge, human reassessment, factual correction and remediation for adverse AI decisions.
- Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
- Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
Explanation
Correct Answers
B:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. For this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
E:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. This response fits the risk associated with Threshold governance and safety fallbacks: Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
Incorrect Answers
A:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
C:This option concerns Customer appeal and remedy for automated outcomes, which is a related but different AI governance concern. In this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
D:This option concerns Feedback-loop contamination from user corrections, which is a related but different AI governance concern. For this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
Question 681
A governance and assurance review uncovers the following: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Provide reviewers timely context, override authority, training and auditable independent decision space.
- Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
- Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
Explanation
Correct Answer
C:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. This is the proportionate control response to Threshold governance and safety fallbacks: Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
Incorrect Answers
A:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. The audit must corroborate: Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
B:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. It does not establish control effectiveness for Threshold governance and safety fallbacks.
D:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. In this case: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions.
Question 682
A risk-based audit engagement examines this operating condition: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Auditability of an AI decision depends on traceable context, not just a final predicted label.
- Thresholds are business control decisions whose error costs vary by context and input quality.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Human-in-the-loop labeling does not create real oversight when reviewers cannot understand or influence outcomes.
Explanation
Correct Answer
B:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. The corrective action is: Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
Incorrect Answers
A:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. The audit must corroborate: Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
C:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. For this situation: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions.
D:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. For this situation: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions.
Question 683
An audit committee asks for a defensible assessment of this situation: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
- Inspect service identities, tool-call logs, policy checks, denial tests and human approval traces.
Explanation
Correct Answer
A:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. In this case: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions.
Incorrect Answers
B:For this situation: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions.
C:This option concerns Feedback-loop contamination from user corrections, which is a related but different AI governance concern. The decisive principle here is: Thresholds are business control decisions whose error costs vary by context and input quality.
D:This option concerns Autonomous agent action-limit supervision, which is a related but different AI governance concern. It does not establish control effectiveness for Threshold governance and safety fallbacks.
Question 684
During an AI controls assessment, the team learns that a transport system uses fixed confidence thresholds despite large shifts in seasonal conditions. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- An authorized reviewer can reconstruct a sampled decision without overcollecting personal data.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Uncertain cases route safely and measured errors remain within approved operational and fairness limits.
- Approved feedback improves measured outcomes without adversarial self-reinforcing error.
Explanation
Correct Answer
C:The reason it is necessary is: Thresholds are business control decisions whose error costs vary by context and input quality. Independent audit evidence includes: Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
Incorrect Answers
A:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. For this situation: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions.
B:It misses the key control principle: Thresholds are business control decisions whose error costs vary by context and input quality.
D:This option concerns Feedback-loop contamination from user corrections, which is a related but different AI governance concern. For this situation: A transport system uses fixed confidence thresholds despite large shifts in seasonal conditions.
Question 685
An independent AI auditor reviews this case: An HR scoring model has no owner monitoring monthly subgroup error after initial approval. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Design actionable challenge, human reassessment, factual correction and remediation for adverse AI decisions.
- Conclude that no AI risk exists as long as the model endpoint responds within its SLA.
- Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
- Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
Explanation
Correct Answer
D:Independent audit evidence includes: Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions. The acceptable verified outcome is: Threshold breaches lead to timely investigation and documented changes to protect affected users.
Incorrect Answers
A:This option concerns Customer appeal and remedy for automated outcomes, which is a related but different AI governance concern. It does not establish control effectiveness for AI post-deployment monitoring and KRI ownership.
B:This attractive shortcut does not resolve the AI post-deployment monitoring and KRI ownership weakness: Conclude that no AI risk exists as long as the model endpoint responds within its SLA.
C:This option concerns Independent validation before model deployment, which is a related but different AI governance concern. It does not establish control effectiveness for AI post-deployment monitoring and KRI ownership.
Question 686
A governance and assurance review uncovers the following: An HR scoring model has no owner monitoring monthly subgroup error after initial approval. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Thresholds are business control decisions whose error costs vary by context and input quality.
- Production AI monitoring must measure harmful outcomes, not merely infrastructure availability.
- Autonomous action risk is driven by actual tool permissions and decision gates, not chatbot prompt wording alone.
- Conclude that no AI risk exists as long as the model endpoint responds within its SLA.
Explanation
Correct Answer
B:Independent audit evidence includes: Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions. The corrective action is: Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
Incorrect Answers
A:This option concerns Threshold governance and safety fallbacks, which is a related but different AI governance concern. It does not establish control effectiveness for AI post-deployment monitoring and KRI ownership.
C:This option concerns Autonomous agent action-limit supervision, which is a related but different AI governance concern. It does not establish control effectiveness for AI post-deployment monitoring and KRI ownership.
D:This attractive shortcut does not resolve the AI post-deployment monitoring and KRI ownership weakness: Conclude that no AI risk exists as long as the model endpoint responds within its SLA. For this situation: An HR scoring model has no owner monitoring monthly subgroup error after initial approval.
Question 687
A risk-based audit engagement examines this operating condition: An HR scoring model has no owner monitoring monthly subgroup error after initial approval. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions.
- Review appeal turnaround, reviewer authority, data correction process, overturned cases and notices.
- Inspect service identities, tool-call logs, policy checks, denial tests and human approval traces.
- Conclude that no AI risk exists as long as the model endpoint responds within its SLA.
Explanation
Correct Answer
A:The acceptable verified outcome is: Threshold breaches lead to timely investigation and documented changes to protect affected users. This evidence directly tests the asserted control for AI post-deployment monitoring and KRI ownership: Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions.
Incorrect Answers
B:This option concerns Customer appeal and remedy for automated outcomes, which is a related but different AI governance concern. For this situation: An HR scoring model has no owner monitoring monthly subgroup error after initial approval.
C:This option concerns Autonomous agent action-limit supervision, which is a related but different AI governance concern. For this situation: An HR scoring model has no owner monitoring monthly subgroup error after initial approval.
D:This attractive shortcut does not resolve the AI post-deployment monitoring and KRI ownership weakness: Conclude that no AI risk exists as long as the model endpoint responds within its SLA. Effective closure requires: Threshold breaches lead to timely investigation and documented changes to protect affected users.
Question 688
An audit committee asks for a defensible assessment of this situation: An HR scoring model has no owner monitoring monthly subgroup error after initial approval. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- An independent challenger confirms specified safety and performance requirements before go-live.
- Uncertain cases route safely and measured errors remain within approved operational and fairness limits.
- Conclude that no AI risk exists as long as the model endpoint responds within its SLA.
- Threshold breaches lead to timely investigation and documented changes to protect affected users.
Explanation
Correct Answer
D:Independent audit evidence includes: Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions. This is an observable validation result for AI post-deployment monitoring and KRI ownership, not just management assurance: Threshold breaches lead to timely investigation and documented changes to protect affected users.
Incorrect Answers
A:This option concerns Independent validation before model deployment, which is a related but different AI governance concern. For this situation: An HR scoring model has no owner monitoring monthly subgroup error after initial approval.
B:This option concerns Threshold governance and safety fallbacks, which is a related but different AI governance concern. For this situation: An HR scoring model has no owner monitoring monthly subgroup error after initial approval.
C:This attractive shortcut does not resolve the AI post-deployment monitoring and KRI ownership weakness: Conclude that no AI risk exists as long as the model endpoint responds within its SLA. It misses the key control principle: Production AI monitoring must measure harmful outcomes, not merely infrastructure availability.
Question 689
During an AI controls assessment, the team learns that a fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Conclude that no AI risk exists as long as the model endpoint responds within its SLA.
- Verify feedback provenance, reviewer authority and sampling before incorporating live corrections into future models.
- Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
- Capture necessary model version, inputs or safe references, key context, outcome and human intervention metadata.
Explanation
Correct Answer
C:Independent audit evidence includes: Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions. In this case: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks.
Incorrect Answers
A:This attractive shortcut does not resolve the AI post-deployment monitoring and KRI ownership weakness: Conclude that no AI risk exists as long as the model endpoint responds within its SLA. In this case: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks.
B:This option concerns Feedback-loop contamination from user corrections, which is a related but different AI governance concern. It does not establish control effectiveness for AI post-deployment monitoring and KRI ownership.
D:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. It does not establish control effectiveness for AI post-deployment monitoring and KRI ownership.
Question 690
An independent AI auditor reviews this case: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
- Limit agent authority by least-privilege tools, transaction ceilings, pre-action approval and post-action verification.
- Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions.
- Inspect logging schema, privacy redaction, identifiers, time ordering, retention and replay tests.
- Conclude that no AI risk exists as long as the model endpoint responds within its SLA.
Explanation
Correct Answers
A:Independent audit evidence includes: Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions. For this situation: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks.
C:The acceptable verified outcome is: Threshold breaches lead to timely investigation and documented changes to protect affected users. In this case: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks.
Incorrect Answers
B:This option concerns Autonomous agent action-limit supervision, which is a related but different AI governance concern. In this case: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks.
D:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. For this situation: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks.
E:This attractive shortcut does not resolve the AI post-deployment monitoring and KRI ownership weakness: Conclude that no AI risk exists as long as the model endpoint responds within its SLA. For this situation: A fraud platform monitors API uptime but not a rising rate of wrongful customer payment blocks.