Isaca AAIA Practice Test 47
Question 460
An independent AI auditor reviews this case: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
- Evaluate class distribution with relevant precision, recall and business costs, and use justified stratification or targeted samples.
- Inspect feature timestamps, label windows, join keys, leakage tests and reconstructed historical decisions.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
Explanation
Correct Answers
A:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
E:This response fits the risk associated with RAG corpus access controls and tenant segregation: Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
Incorrect Answers
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
C:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. For this situation: A customer-support RAG assistant retrieves a private client's contract while answering another client's generic question.
D:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. Effective closure requires: Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
Question 461
A governance and assurance review uncovers the following: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
- Validate synthetic data for utility, subgroup representation, memorization and plausible disclosure before release.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Rate-limit and monitor queries, restrict unnecessary output detail and evaluate extraction risks proportionate to value.
Explanation
Correct Answer
A:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. In this case: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated.
Incorrect Answers
B:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. It does not establish control effectiveness for RAG corpus access controls and tenant segregation.
C:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. In this case: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated.
D:This option concerns AI model extraction and excessive query access, which is a related but different AI governance concern. It does not establish control effectiveness for RAG corpus access controls and tenant segregation.
Question 462
A risk-based audit engagement examines this operating condition: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Training-data integrity is a control boundary because malicious or flawed inputs can change model behavior persistently.
- Accuracy on imbalanced data can obscure missed rare high-consequence cases or overwhelming false positives.
- Private text can leak through retrieval even if the visible application screen enforces role-based access.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
Explanation
Correct Answer
C:The acceptable verified outcome is: Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged. In this case: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated.
Incorrect Answers
A:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. The decisive principle here is: Private text can leak through retrieval even if the visible application screen enforces role-based access.
B:This option concerns Class imbalance and minority event coverage, which is a related but different AI governance concern. The decisive principle here is: Private text can leak through retrieval even if the visible application screen enforces role-based access.
D:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. For this situation: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated.
Question 463
An audit committee asks for a defensible assessment of this situation: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Inspect authentication, rate limits, output granularity, query anomalies and abuse response.
Explanation
Correct Answer
B:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. This evidence directly tests the asserted control for RAG corpus access controls and tenant segregation: Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Incorrect Answers
A:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The decisive principle here is: Private text can leak through retrieval even if the visible application screen enforces role-based access.
D:This option concerns AI model extraction and excessive query access, which is a related but different AI governance concern. For this situation: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated.
Question 464
During an AI controls assessment, the team learns that an internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Suspicious extraction patterns are detected and unauthorized systematic probing is constrained.
- An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
- Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
Explanation
Correct Answer
D:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. This is an observable validation result for RAG corpus access controls and tenant segregation, not just management assurance: Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
Incorrect Answers
A:For this situation: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated.
B:This option concerns AI model extraction and excessive query access, which is a related but different AI governance concern. The decisive principle here is: Private text can leak through retrieval even if the visible application screen enforces role-based access.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. For this situation: An internal knowledge bot searches a global embedding store where permissions are checked only after an answer is generated.
Question 465
An independent AI auditor reviews this case: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Rate-limit and monitor queries, restrict unnecessary output detail and evaluate extraction risks proportionate to value.
- Validate provenance and anomaly patterns for incoming data, quarantine suspicious updates and approve retraining sources.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
Explanation
Correct Answer
D:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. This is the proportionate control response to RAG corpus access controls and tenant segregation: Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
Incorrect Answers
A:This option concerns AI model extraction and excessive query access, which is a related but different AI governance concern. The audit must corroborate: Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
B:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. The audit must corroborate: Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
C:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. In this case: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement.
Question 466
A governance and assurance review uncovers the following: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Synthetic labels do not automatically eliminate privacy, bias or representativeness risk.
- Private text can leak through retrieval even if the visible application screen enforces role-based access.
- Future information leakage creates unrealistic validation metrics and cannot be tolerated in prospective inference.
Explanation
Correct Answer
C:The acceptable verified outcome is: Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged. Independent audit evidence includes: Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Incorrect Answers
A:This attractive shortcut does not resolve the RAG corpus access controls and tenant segregation weakness: Rely on the language model to politely ignore confidential chunks already placed in its context. For this situation: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement.
B:This option concerns Synthetic training data utility and privacy controls, which is a related but different AI governance concern. The decisive principle here is: Private text can leak through retrieval even if the visible application screen enforces role-based access.
D:This option concerns Feature-store time leakage in model evaluation, which is a related but different AI governance concern. The decisive principle here is: Private text can leak through retrieval even if the visible application screen enforces role-based access.
Question 467
A risk-based audit engagement examines this operating condition: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Review label guidelines, inter-rater reliability, adjudication logs, sampled errors and retraining effects.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Explanation
Correct Answer
D:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. The corrective action is: Enforce identity-aware document retrieval and tenant boundaries before context enters the model.
Incorrect Answers
A:It misses the key control principle: Private text can leak through retrieval even if the visible application screen enforces role-based access.
B:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. The decisive principle here is: Private text can leak through retrieval even if the visible application screen enforces role-based access.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The audit must corroborate: Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Question 468
An audit committee asks for a defensible assessment of this situation: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Labels are consistent against approved definitions, and model tests expose residual uncertain classes.
- Rely on the language model to politely ignore confidential chunks already placed in its context.
- Suspicious extraction patterns are detected and unauthorized systematic probing is constrained.
- Authorized queries work while cross-tenant and restricted-content retrievals are blocked and logged.
Explanation
Correct Answer
D:The reason it is necessary is: Private text can leak through retrieval even if the visible application screen enforces role-based access. Independent audit evidence includes: Inspect index ACL propagation, chunk metadata, identity claims, denied-search tests and cached answer behavior.
Incorrect Answers
A:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. For this situation: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement.
B:For this situation: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement.
C:This option concerns AI model extraction and excessive query access, which is a related but different AI governance concern. For this situation: A law firm combines restricted matter documents in a shared vector index with no per-user retrieval enforcement.
Question 469
During an AI controls assessment, the team learns that a healthcare team uses synthetic patient rows but fails membership tests that reveal copied rare patient combinations. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
- Test extractable memorization and membership inference under authorized red-team conditions and apply data minimization.
- Label a table synthetic and distribute it publicly without any disclosure or realism assessment.
- Validate synthetic data for utility, subgroup representation, memorization and plausible disclosure before release.
Explanation
Correct Answer
D:The reason it is necessary is: Synthetic labels do not automatically eliminate privacy, bias or representativeness risk. The acceptable verified outcome is: Synthetic samples add task value within approved privacy and fairness criteria with bounded memorization risk.
Incorrect Answers
A:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. It does not establish control effectiveness for Synthetic training data utility and privacy controls.
B:This option concerns Membership inference and training-data memorization tests, which is a related but different AI governance concern. It does not establish control effectiveness for Synthetic training data utility and privacy controls.
C:This attractive shortcut does not resolve the Synthetic training data utility and privacy controls weakness: Label a table synthetic and distribute it publicly without any disclosure or realism assessment.
Question 470
An independent AI auditor reviews this case: A healthcare team uses synthetic patient rows but fails membership tests that reveal copied rare patient combinations. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Compare ingestion source identities, sampling patterns, hash manifests, anomaly screens and approval records.
- Validate synthetic data for utility, subgroup representation, memorization and plausible disclosure before release.
- Inspect generation method, similarity and disclosure tests, subgroup distributions and downstream use-case validation.
- Label a table synthetic and distribute it publicly without any disclosure or realism assessment.
- Define annotation rubrics, representative blind review, disagreement resolution and documented quality thresholds.
Explanation
Correct Answers
B:The reason it is necessary is: Synthetic labels do not automatically eliminate privacy, bias or representativeness risk. For this situation: A healthcare team uses synthetic patient rows but fails membership tests that reveal copied rare patient combinations.
C:The reason it is necessary is: Synthetic labels do not automatically eliminate privacy, bias or representativeness risk. In this case: A healthcare team uses synthetic patient rows but fails membership tests that reveal copied rare patient combinations.
Incorrect Answers
A:This option concerns Data poisoning prevention in automated ingestion, which is a related but different AI governance concern. It does not establish control effectiveness for Synthetic training data utility and privacy controls.
D:This attractive shortcut does not resolve the Synthetic training data utility and privacy controls weakness: Label a table synthetic and distribute it publicly without any disclosure or realism assessment. For this situation: A healthcare team uses synthetic patient rows but fails membership tests that reveal copied rare patient combinations.
E:This option concerns Label quality and human annotator agreement, which is a related but different AI governance concern. For this situation: A healthcare team uses synthetic patient rows but fails membership tests that reveal copied rare patient combinations.