Isaca AAIA Practice Test 46
Question 450
An independent AI auditor reviews this case: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect generation method, similarity and disclosure tests, subgroup distributions and downstream use-case validation.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Accept impressive validation results without checking whether features encode outcomes from the future.
- Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
Explanation
Correct Answers
D:This response fits the risk associated with Feature-store time leakage in model evaluation: Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
E:The reason it is necessary is: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference. The corrective action is: Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
Incorrect Answers
A:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. It does not establish control effectiveness for Feature-store time leakage in model evaluation.
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. In this case: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions.
C:This attractive shortcut does not resolve the Feature-store time leakage in model evaluation weakness: Accept impressive validation results without checking whether features encode outcomes from the future. For this situation: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions.
Question 451
A governance and assurance review uncovers the following: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Accept impressive validation results without checking whether features encode outcomes from the future.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Explanation
Correct Answer
A:The reason it is necessary is: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference. For this situation: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions.
Incorrect Answers
B:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. It does not establish control effectiveness for Feature-store time leakage in model evaluation.
C:This attractive shortcut does not resolve the Feature-store time leakage in model evaluation weakness: Accept impressive validation results without checking whether features encode outcomes from the future. Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
D:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. The decisive principle here is: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference.
Question 452
A risk-based audit engagement examines this operating condition: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Accept impressive validation results without checking whether features encode outcomes from the future.
- Held-out evaluation uses only information available at the decision time and predicts realistic production performance.
- An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
- Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
Explanation
Correct Answer
B:The reason it is necessary is: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference. Independent audit evidence includes: Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
Incorrect Answers
A:For this situation: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. For this situation: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions.
D:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. For this situation: A medical risk forecast uses post-discharge records that would be unknown when clinicians originally made decisions.
Question 453
An audit committee asks for a defensible assessment of this situation: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
- Accept impressive validation results without checking whether features encode outcomes from the future.
- Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
- Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
Explanation
Correct Answer
C:The reason it is necessary is: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference. In this case: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
Incorrect Answers
A:This option concerns RAG corpus access controls and tenant segregation, which is a related but different AI governance concern. In this case: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
B:This attractive shortcut does not resolve the Feature-store time leakage in model evaluation weakness: Accept impressive validation results without checking whether features encode outcomes from the future. In this case: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
D:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The decisive principle here is: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference.
Question 454
During an AI controls assessment, the team learns that an insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Training-data integrity is a control boundary because malicious or flawed inputs can change model behavior persistently.
- AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
- Accept impressive validation results without checking whether features encode outcomes from the future.
- Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference.
Explanation
Correct Answer
D:The acceptable verified outcome is: Held-out evaluation uses only information available at the decision time and predicts realistic production performance. In this case: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
Incorrect Answers
A:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. In this case: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
B:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. For this situation: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
C:This attractive shortcut does not resolve the Feature-store time leakage in model evaluation weakness: Accept impressive validation results without checking whether features encode outcomes from the future. For this situation: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
Question 455
An independent AI auditor reviews this case: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Accept impressive validation results without checking whether features encode outcomes from the future.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
Explanation
Correct Answer
C:The acceptable verified outcome is: Held-out evaluation uses only information available at the decision time and predicts realistic production performance.
Incorrect Answers
A:For this situation: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
B:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The audit must corroborate: Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
D:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots.
Question 456
A governance and assurance review uncovers the following: An insurer scores a time-split validation using current customer attributes instead of point-in-time snapshots. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
- Held-out evaluation uses only information available at the decision time and predicts realistic production performance.
- Synthetic samples add task value within approved privacy and fairness criteria with bounded memorization risk.
- Accept impressive validation results without checking whether features encode outcomes from the future.
Explanation
Correct Answer
B:This is an observable validation result for Feature-store time leakage in model evaluation, not just management assurance: Held-out evaluation uses only information available at the decision time and predicts realistic production performance.
Incorrect Answers
A:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The outcome to verify is: Held-out evaluation uses only information available at the decision time and predicts realistic production performance.
C:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. The decisive principle here is: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference.
D:It misses the key control principle: Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference.
Question 457
A risk-based audit engagement examines this operating condition: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Enforce point-in-time joins and temporal cutoffs for training and evaluation features.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
Explanation
Correct Answer
C:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. The acceptable verified outcome is: Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
Incorrect Answers
A:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. It does not establish control effectiveness for RAG corpus access controls and tenant segregation.
B:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. It does not establish control effectiveness for RAG corpus access controls and tenant segregation.
D:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context.
Question 458
An audit committee asks for a defensible assessment of this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Training-data integrity is a control boundary because malicious or flawed inputs can change model behavior persistently.
- Private text can leak through retrieval even if the visible application screen enforces role-based access.
- AI outputs cannot be audited reliably when their source records, transformations and model lineage cannot be reconstructed.
Explanation
Correct Answer
C:The acceptable verified outcome is: Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged. This is the applicable AI audit principle for RAG corpus access controls and tenant segregation: Private text can leak through retrieval even if the visible application screen enforces role-based access.
Incorrect Answers
A:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
B:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. It does not establish control effectiveness for RAG corpus access controls and tenant segregation.
D:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. It does not establish control effectiveness for RAG corpus access controls and tenant segregation.
Question 459
During an AI controls assessment, the team learns that a customer-support RAG assistant retrieves a private client's contract while answering another client's generic question. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
- Inspect confusion matrices, base rates, stratified errors, class weighting choices and cost assumptions.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
Explanation
Correct Answer
A:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. In this case: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
Incorrect Answers
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. It does not establish control effectiveness for RAG corpus access controls and tenant segregation.
C:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
D:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. The auditor should instead examine: Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Question 460
An independent AI auditor reviews this case: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
Explanation
Correct Answers
A:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
E:This response fits the risk associated with RAG corpus access controls and tenant segregation: Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
Incorrect Answers
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
C:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
D:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. Effective closure requires: Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
