Isaca AAIA Practice Test 41
Question 400
An independent AI auditor reviews this case: A bank's fraud model changes after a retraining run but nobody can identify which customer data snapshot generated the weights. Which THREE elements together provide a defensible AI audit conclusion: the corrective control, the evidence to test it, and a verified outcome? Select THREE answers.
- Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
- Use independent held-out evaluations with leakage checks and repeatable challenge sets not tuned into training.
- Treat the current live database as proof of what the model saw during a prior training run.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
Explanation
Correct Answers
A:The reason it is necessary is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed. For this situation: A bank's fraud model changes after a retraining run but nobody can identify which customer data snapshot generated the weights.
E:The reason it is necessary is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed. This evidence directly tests the asserted control for AI training dataset lineage and reproducibility: Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
F:The reason it is necessary is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed. Independent audit evidence includes: Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
Incorrect Answers
B:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: A bank's fraud model changes after a retraining run but nobody can identify which customer data snapshot generated the weights.
C:This option concerns Training and evaluation benchmark contamination, which is a related but different AI governance concern. For this situation: A bank's fraud model changes after a retraining run but nobody can identify which customer data snapshot generated the weights.
D:This attractive shortcut does not resolve the AI training dataset lineage and reproducibility weakness: Treat the current live database as proof of what the model saw during a prior training run. Effective closure requires: An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
Question 401
A governance and assurance review uncovers the following: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
- Validate synthetic data for utility, subgroup representation, memorization and plausible disclosure before release.
- Treat the current live database as proof of what the model saw during a prior training run.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Explanation
Correct Answer
A:The reason it is necessary is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed. In this case: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage.
Incorrect Answers
B:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. It does not establish control effectiveness for AI training dataset lineage and reproducibility.
C:This attractive shortcut does not resolve the AI training dataset lineage and reproducibility weakness: Treat the current live database as proof of what the model saw during a prior training run. In this case: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage.
D:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. The decisive principle here is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
Question 402
A risk-based audit engagement examines this operating condition: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
- Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
- Synthetic labels do not automatically eliminate privacy, bias or representativeness risk.
- Treat the current live database as proof of what the model saw during a prior training run.
Explanation
Correct Answer
A:The acceptable verified outcome is: An independent rerun can identify the exact source data and material transformation decisions for a sampled model version. Independent audit evidence includes: Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
Incorrect Answers
B:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. For this situation: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage.
C:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. The decisive principle here is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
D:This attractive shortcut does not resolve the AI training dataset lineage and reproducibility weakness: Treat the current live database as proof of what the model saw during a prior training run. For this situation: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage.
Question 403
An audit committee asks for a defensible assessment of this situation: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Treat the current live database as proof of what the model saw during a prior training run.
Explanation
Correct Answer
C:The reason it is necessary is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed. The corrective action is: Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
Incorrect Answers
A:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. The decisive principle here is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
B:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. The decisive principle here is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
D:This attractive shortcut does not resolve the AI training dataset lineage and reproducibility weakness: Treat the current live database as proof of what the model saw during a prior training run. Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
Question 404
During an AI controls assessment, the team learns that a hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
- Treat the current live database as proof of what the model saw during a prior training run.
- Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
- Performance advantage remains on independent representative evaluation with minimal leakage.
Explanation
Correct Answer
A:The reason it is necessary is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed. This is an observable validation result for AI training dataset lineage and reproducibility, not just management assurance: An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
Incorrect Answers
B:It misses the key control principle: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
C:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. For this situation: A hospital model card cites a data version that was overwritten by the latest ETL job with no immutable lineage.
D:This option concerns Training and evaluation benchmark contamination, which is a related but different AI governance concern. The decisive principle here is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
Question 405
An independent AI auditor reviews this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Treat the current live database as proof of what the model saw during a prior training run.
- Validate synthetic data for utility, subgroup representation, memorization and plausible disclosure before release.
Explanation
Correct Answer
A:The reason it is necessary is: AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed. In this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
Incorrect Answers
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. It does not establish control effectiveness for AI training dataset lineage and reproducibility.
C:This attractive shortcut does not resolve the AI training dataset lineage and reproducibility weakness: Treat the current live database as proof of what the model saw during a prior training run. In this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
D:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. In this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
Question 406
A governance and assurance review uncovers the following: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Benchmark scores can overstate generalization when test items or related records have been exposed to development.
- AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
- Treat the current live database as proof of what the model saw during a prior training run.
- Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated.
Explanation
Correct Answer
B:The acceptable verified outcome is: An independent rerun can identify the exact source data and material transformation decisions for a sampled model version. In this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
Incorrect Answers
A:This option concerns Training and evaluation benchmark contamination, which is a related but different AI governance concern. In this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
C:This attractive shortcut does not resolve the AI training dataset lineage and reproducibility weakness: Treat the current live database as proof of what the model saw during a prior training run. For this situation: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
D:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. In this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
Question 407
A risk-based audit engagement examines this operating condition: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
- Treat the current live database as proof of what the model saw during a prior training run.
Explanation
Correct Answer
B:The acceptable verified outcome is: An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
Incorrect Answers
A:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. For this situation: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
C:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. For this situation: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
D:For this situation: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
Question 408
An audit committee asks for a defensible assessment of this situation: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Synthetic samples add task value within approved privacy and fairness criteria with bounded memorization risk.
- An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
- Treat the current live database as proof of what the model saw during a prior training run.
- Performance advantage remains on independent representative evaluation with minimal leakage.
Explanation
Correct Answer
B:In this case: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
Incorrect Answers
A:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. For this situation: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
C:The auditor should instead examine: Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
D:This option concerns Training and evaluation benchmark contamination, which is a related but different AI governance concern. For this situation: A product team cannot reproduce an AI error because external feature feeds have been replaced without historical snapshots.
Question 409
During an AI controls assessment, the team learns that a claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Provide reviewers timely context, override authority, training and auditable independent decision space.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
Explanation
Correct Answer
B:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. This is the proportionate control response to Label quality and human annotator agreement: Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Incorrect Answers
A:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
C:This option concerns Effective human oversight of AI recommendations, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
D:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
Question 410
An independent AI auditor reviews this case: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
- Validate provenance and anomaly patterns for incoming data, quarantine suspicious updates and approve retraining sources.
Explanation
Correct Answers
A:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. This evidence directly tests the asserted control for Label quality and human annotator agreement: Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
B:The reason it is necessary is: Garbage or inconsistent ground-truth labels cap model validity even if the learning algorithm is sophisticated. This response fits the risk associated with Label quality and human annotator agreement: Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Incorrect Answers
C:This attractive shortcut does not resolve the Label quality and human annotator agreement weakness: Use majority vote from untrained annotators as conclusive truth without evaluating disputed cases. For this situation: A claims classifier is trained on insurance fraud labels supplied by contractors whose disagreements are never measured.
D:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
E:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. It does not establish control effectiveness for Label quality and human annotator agreement.
