Isaca AAIA Practice Test 43
Question 420
An independent AI auditor reviews this case: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
Explanation
Correct Answers
A:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
D:This response fits the risk associated with Label quality and human annotator agreement: Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Incorrect Answers
B:It misses the key control principle: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
C:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
E:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. For this situation: An image defect model's data labels change based on who is on shift, but no adjudication rules are documented.
Question 421
A governance and assurance review uncovers the following: A chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Explanation
Correct Answer
B:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. In this case: A chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate.
Incorrect Answers
A:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. It does not establish control effectiveness for Class imbalance and minority event coverage.
C:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses.
D:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. It does not establish control effectiveness for Class imbalance and minority event coverage.
Question 422
A risk-based audit engagement examines this operating condition: A chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Private text can leak through retrieval even if the visible application screen enforces role-based access.
- Training-data integrity is a control boundary because malicious or flawed inputs can change model behavior persistently.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
Explanation
Correct Answer
D:Independent audit evidence includes: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions. This is the applicable AI audit principle for Class imbalance and minority event coverage: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
Incorrect Answers
A:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. It does not establish control effectiveness for Class imbalance and minority event coverage.
B:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. It does not establish control effectiveness for Class imbalance and minority event coverage.
C:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. For this situation: A chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate.
Question 423
An audit committee asks for a defensible assessment of this situation: A chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
Explanation
Correct Answer
B:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. This evidence directly tests the asserted control for Class imbalance and minority event coverage: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
Incorrect Answers
A:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. The auditor should instead examine: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
C:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. For this situation: A chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate.
D:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. It does not establish control effectiveness for Class imbalance and minority event coverage.
Question 424
During an AI controls assessment, the team learns that a chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Held-out evaluation uses only information available at the decision time and predicts realistic production performance.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Threshold and sample design meet documented detection and false-intervention goals for rare events.
- Synthetic samples add task value within approved privacy and fairness criteria with bounded memorization risk.
Explanation
Correct Answer
C:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. This is an observable validation result for Class imbalance and minority event coverage, not just management assurance: Threshold and sample design meet documented detection and false-intervention goals for rare events.
Incorrect Answers
A:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. For this situation: A chargeback model reports 99 percent accuracy because nearly all transactions in its test set are legitimate.
B:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. Effective closure requires: Threshold and sample design meet documented detection and false-intervention goals for rare events.
D:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. It does not establish control effectiveness for Class imbalance and minority event coverage.
Question 425
An independent AI auditor reviews this case: A fraud classifier catches many common cases but misses almost every rare account takeover event. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Create traceable business, human impact, security and quality requirements with measurable release and acceptance gates.
Explanation
Correct Answer
C:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. In this case: A fraud classifier catches many common cases but misses almost every rare account takeover event.
Incorrect Answers
A:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. The decisive principle here is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
B:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. In this case: A fraud classifier catches many common cases but misses almost every rare account takeover event.
D:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. It does not establish control effectiveness for Class imbalance and minority event coverage.
Question 426
A governance and assurance review uncovers the following: A fraud classifier catches many common cases but misses almost every rare account takeover event. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Training-data integrity is a control boundary because malicious or flawed inputs can change model behavior persistently.
- Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
- Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
Explanation
Correct Answer
B:Independent audit evidence includes: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions. In this case: A fraud classifier catches many common cases but misses almost every rare account takeover event.
Incorrect Answers
A:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. The decisive principle here is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
C:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. The decisive principle here is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
D:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. For this situation: A fraud classifier catches many common cases but misses almost every rare account takeover event.
Question 427
A risk-based audit engagement examines this operating condition: A fraud classifier catches many common cases but misses almost every rare account takeover event. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Inspect generation method, similarity and disclosure tests, subgroup distributions and downstream use-case validation.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
Explanation
Correct Answer
D:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. The corrective action is: Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
Incorrect Answers
A:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
B:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. The decisive principle here is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The decisive principle here is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
Question 428
An audit committee asks for a defensible assessment of this situation: A fraud classifier catches many common cases but misses almost every rare account takeover event. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Contaminated feeds are contained and verified clean data can be restored without corrupting promoted models.
- Threshold and sample design meet documented detection and false-intervention goals for rare events.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
Explanation
Correct Answer
B:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. Independent audit evidence includes: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
Incorrect Answers
A:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: A fraud classifier catches many common cases but misses almost every rare account takeover event.
C:For this situation: A fraud classifier catches many common cases but misses almost every rare account takeover event.
D:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. For this situation: A fraud classifier catches many common cases but misses almost every rare account takeover event.
Question 429
During an AI controls assessment, the team learns that a security anomaly detector uses random evaluation samples that contain too few confirmed attacks to measure recall. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Validate provenance and anomaly patterns for incoming data, quarantine suspicious updates and approve retraining sources.
- Validate synthetic data for utility, subgroup representation, memorization and plausible disclosure before release.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
Explanation
Correct Answer
A:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. This is the proportionate control response to Class imbalance and minority event coverage: Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
Incorrect Answers
B:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. The audit must corroborate: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
C:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. The audit must corroborate: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
D:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. In this case: A security anomaly detector uses random evaluation samples that contain too few confirmed attacks to measure recall.
Question 430
An independent AI auditor reviews this case: A security anomaly detector uses random evaluation samples that contain too few confirmed attacks to measure recall. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
- Select the model with the highest overall accuracy while ignoring minority-case misses.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
Explanation
Correct Answers
A:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. The acceptable verified outcome is: Threshold and sample design meet documented detection and false-intervention goals for rare events.
E:The reason it is necessary is: Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives. This response fits the risk associated with Class imbalance and minority event coverage: Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
Incorrect Answers
B:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. The audit must corroborate: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The audit must corroborate: Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
D:This attractive shortcut does not resolve the Class imbalance and minority event coverage weakness: Select the model with the highest overall accuracy while ignoring minority-case misses. For this situation: A security anomaly detector uses random evaluation samples that contain too few confirmed attacks to measure recall.
