Isaca AAIA Practice Test 68
Question 670
An independent AI auditor reviews this case: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
- Verify feedback provenance, reviewer authority and sampling before incorporating live corrections into future models.
- Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
- Review appeal turnaround, reviewer authority, data correction process, overturned cases and notices.
- Treat every user click or thumbs-up as an authoritative label and retrain automatically.
Explanation
Correct Answers
A:The acceptable verified outcome is: Approved feedback improves measured outcomes without adversarial self-reinforcing error.
B:The reason it is necessary is: Feedback improves models only when its quality and susceptibility to manipulation are controlled. For this situation: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels.
Incorrect Answers
C:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. For this situation: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels.
D:This option concerns Customer appeal and remedy for automated outcomes, which is a related but different AI governance concern. For this situation: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels.
E:This attractive shortcut does not resolve the Feedback-loop contamination from user corrections weakness: Treat every user click or thumbs-up as an authoritative label and retrain automatically. For this situation: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels.
Question 671
A governance and assurance review uncovers the following: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect service identities, tool-call logs, policy checks, denial tests and human approval traces.
- Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
- Treat every user click or thumbs-up as an authoritative label and retrain automatically.
- Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions.
Explanation
Correct Answer
B:For this situation: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels.
Incorrect Answers
A:This option concerns Autonomous agent action-limit supervision, which is a related but different AI governance concern. The audit must corroborate: Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
C:The auditor should instead examine: Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
D:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. The audit must corroborate: Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
Question 672
A risk-based audit engagement examines this operating condition: A medical classifier ingests clinician corrections that contain unresolved disagreements as verified labels. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Threshold breaches lead to timely investigation and documented changes to protect affected users.
- An authorized reviewer can reconstruct a sampled decision without overcollecting personal data.
- Approved feedback improves measured outcomes without adversarial self-reinforcing error.
- Treat every user click or thumbs-up as an authoritative label and retrain automatically.
Explanation
Correct Answer
C:This is an observable validation result for Feedback-loop contamination from user corrections, not just management assurance: Approved feedback improves measured outcomes without adversarial self-reinforcing error.
Incorrect Answers
A:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. The outcome to verify is: Approved feedback improves measured outcomes without adversarial self-reinforcing error.
B:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. The audit must corroborate: Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
D:Effective closure requires: Approved feedback improves measured outcomes without adversarial self-reinforcing error.
Question 673
An audit committee asks for a defensible assessment of this situation: A hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Provide reviewers timely context, override authority, training and auditable independent decision space.
- Capture necessary model version, inputs or safe references, key context, outcome and human intervention metadata.
Explanation
Correct Answer
A:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. In this case: A hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity.
Incorrect Answers
B:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
C:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. It does not establish control effectiveness for Threshold governance and safety fallbacks.
D:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. It does not establish control effectiveness for Threshold governance and safety fallbacks.
Question 674
During an AI controls assessment, the team learns that a hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Feedback improves models only when its quality and susceptibility to manipulation are controlled.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Thresholds are business control decisions whose error costs vary by context and input quality.
- Production AI monitoring must measure harmful outcomes, not merely infrastructure availability.
Explanation
Correct Answer
C:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. This is the applicable AI audit principle for Threshold governance and safety fallbacks: Thresholds are business control decisions whose error costs vary by context and input quality.
Incorrect Answers
A:This option concerns Feedback-loop contamination from user corrections, which is a related but different AI governance concern. It does not establish control effectiveness for Threshold governance and safety fallbacks.
B:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. For this situation: A hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity.
D:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. It does not establish control effectiveness for Threshold governance and safety fallbacks.
Question 675
An independent AI auditor reviews this case: A hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Review appeal turnaround, reviewer authority, data correction process, overturned cases and notices.
- Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
- Inspect logging schema, privacy redaction, identifiers, time ordering, retention and replay tests.
Explanation
Correct Answer
C:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. This evidence directly tests the asserted control for Threshold governance and safety fallbacks: Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
Incorrect Answers
A:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. It misses the key control principle: Thresholds are business control decisions whose error costs vary by context and input quality.
B:This option concerns Customer appeal and remedy for automated outcomes, which is a related but different AI governance concern. It does not establish control effectiveness for Threshold governance and safety fallbacks.
D:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. For this situation: A hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity.
Question 676
A governance and assurance review uncovers the following: A hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Reviewers correctly reject unsafe recommendations and escalation works within operational time constraints.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Uncertain cases route safely and measured errors remain within approved operational and fairness limits.
- An authorized reviewer can reconstruct a sampled decision without overcollecting personal data.
Explanation
Correct Answer
C:The reason it is necessary is: Thresholds are business control decisions whose error costs vary by context and input quality. This is an observable validation result for Threshold governance and safety fallbacks, not just management assurance: Uncertain cases route safely and measured errors remain within approved operational and fairness limits.
Incorrect Answers
A:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. For this situation: A hospital AI auto-escalates every score over 0.5 without validating calibration or clinical capacity.
B:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. The auditor should instead examine: Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
D:This option concerns AI decision logs and explanation traceability, which is a related but different AI governance concern. The decisive principle here is: Thresholds are business control decisions whose error costs vary by context and input quality.
Question 677
A risk-based audit engagement examines this operating condition: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Provide reviewers timely context, override authority, training and auditable independent decision space.
- Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
- Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
Explanation
Correct Answer
C:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. In this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
Incorrect Answers
A:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. In this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
B:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. In this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
D:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. In this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
Question 678
An audit committee asks for a defensible assessment of this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Thresholds are business control decisions whose error costs vary by context and input quality.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Human-in-the-loop labeling does not create real oversight when reviewers cannot understand or influence outcomes.
- Production AI monitoring must measure harmful outcomes, not merely infrastructure availability.
Explanation
Correct Answer
A:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. Independent audit evidence includes: Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
Incorrect Answers
B:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. For this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
C:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. For this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
D:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. For this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
Question 679
During an AI controls assessment, the team learns that a fraud model blocks payments when input features are missing rather than invoking a safer manual review path. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
- Inspect reviewer workload, UI, explanations, override log and interviews on actual cases.
Explanation
Correct Answer
C:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. The reason it is necessary is: Thresholds are business control decisions whose error costs vary by context and input quality.
Incorrect Answers
A:This option concerns Feedback-loop contamination from user corrections, which is a related but different AI governance concern. In this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
B:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. Effective closure requires: Uncertain cases route safely and measured errors remain within approved operational and fairness limits.
D:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. The decisive principle here is: Thresholds are business control decisions whose error costs vary by context and input quality.
Question 680
An independent AI auditor reviews this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population.
- Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
- Design actionable challenge, human reassessment, factual correction and remediation for adverse AI decisions.
- Inspect feedback source identities, adjudication rates, unusual bursts, retraining eligibility and rollback.
- Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
Explanation
Correct Answers
B:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. For this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
E:The acceptable verified outcome is: Uncertain cases route safely and measured errors remain within approved operational and fairness limits. This response fits the risk associated with Threshold governance and safety fallbacks: Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
Incorrect Answers
A:This attractive shortcut does not resolve the Threshold governance and safety fallbacks weakness: Use an arbitrary raw score threshold and assume model confidence has the same meaning for every population. Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
C:This option concerns Customer appeal and remedy for automated outcomes, which is a related but different AI governance concern. In this case: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.
D:This option concerns Feedback-loop contamination from user corrections, which is a related but different AI governance concern. For this situation: A fraud model blocks payments when input features are missing rather than invoking a safer manual review path.