Isaca AAIA Practice Test 42
Question 410
An independent AI auditor reviews this case: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
- Validate provenance and anomaly patterns for incoming data, quarantine suspicious updates and approve retraining sources.
Explanation
Correct Answers
A:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. This evidence directly tests the asserted control for Label quality and human annotator agreement: Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
B:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. This response fits the risk associated with Label quality and human annotator agreement: Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Incorrect Answers
C:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. For this situation: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured.
D:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
E:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
Question 411
A governance and assurance review uncovers the following: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
Explanation
Correct Answer
D:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. For this situation: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured.
Incorrect Answers
A:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured.
B:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. The auditor should instead examine: Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
C:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. For this situation: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured.
Question 412
A risk-based audit engagement examines this operating condition: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
- Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Synthetic samples add task value within approved privacy and fairness criteria with bounded memorization risk.
Explanation
Correct Answer
B:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. This is an observable validation result for Label quality and human annotator agreement, not just management assurance: Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
Incorrect Answers
A:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
C:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. Effective closure requires: Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
D:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
Question 413
An audit committee asks for a defensible assessment of this situation: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Validate provenance and anomaly patterns for incoming data, quarantine suspicious updates and approve retraining sources.
Explanation
Correct Answer
C:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. Independent audit evidence includes: Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
Incorrect Answers
A:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. In this case: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples.
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. The decisive principle here is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
D:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. The decisive principle here is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
Question 414
During an AI controls assessment, the team learns that a content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Synthetic labels do not automatically eliminate privacy, bias or representativeness risk.
- Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
Explanation
Correct Answer
D:Independent audit evidence includes: Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects. This is the applicable AI audit principle for Label quality and human annotator agreement: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
Incorrect Answers
A:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. For this situation: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples.
B:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. For this situation: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples.
C:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. The decisive principle here is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
Question 415
An independent AI auditor reviews this case: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Inspect reviewer workload, UI, explanations, override log and interviews on actual cases.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Explanation
Correct Answer
C:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. The corrective action is: Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Incorrect Answers
A:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
B:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. The decisive principle here is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
D:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. The decisive principle here is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
Question 416
A governance and assurance review uncovers the following: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Contaminated feeds are contained and verified clean data can be restored without corrupting promoted models.
- Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
Explanation
Correct Answer
D:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. In this case: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples.
Incorrect Answers
A:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples.
B:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. For this situation: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples.
C:For this situation: A content moderation model uses unreviewed ambiguous categories and error patterns worsen for non-English examples.
Question 417
A risk-based audit engagement examines this operating condition: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Validate synthetic data for utility, subgroup representation, memorization and plausible disclosure before release.
Explanation
Correct Answer
C:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
Incorrect Answers
A:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
D:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
Question 418
An audit committee asks for a defensible assessment of this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Human-in-the-loop labeling does not create real oversight when reviewers cannot understand or influence outcomes.
- Training-data integrity is a control boundary because malicious or flawed inputs can change model behavior persistently.
- Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
Explanation
Correct Answer
D:Independent audit evidence includes: Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
Incorrect Answers
A:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
B:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
C:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
Question 419
During an AI controls assessment, the team learns that an image defect model's data labels change based on who is on shift, but no adjudication rules are documented. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
- Inspect reviewer workload, UI, explanations, override log and interviews on actual cases.
Explanation
Correct Answer
B:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. The acceptable verified outcome is: Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
Incorrect Answers
A:For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
C:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. In this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
D:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
Question 420
An independent AI auditor reviews this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
Explanation
Correct Answers
A:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
D:This response fits the risk associated with Label quality and human annotator agreement: Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Incorrect Answers
B:It misses the key control principle: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
C:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
E:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
