Isaca AAIA Practice Test 51
Question 500
An independent AI auditor reviews this case: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version. Which THREE elements together provide a defensible AI audit conclusion: the corrective control, the evidence to test it, and a verified outcome? Select THREE answers.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Review endpoint inventory, credentials, retained artifacts, disposal approvals, legal holds and restoration paths.
- Approved model versions can be traced and materially reproduced with recorded dependencies.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Create traceable business, human impact, security and quality requirements with measurable release and acceptance gates.
- Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Explanation
Correct Answers
C:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. Independent audit evidence includes: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
D:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. This response fits the risk associated with Reproducible model training and configuration management: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
F:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. For this situation: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version.
Incorrect Answers
A:For this situation: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version.
B:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
E:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. The audit must corroborate: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Question 501
A governance and assurance review uncovers the following: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Version and trace training, validation and inference datasets with accountable lineage, hashes and reproducible pipeline metadata.
- Separate development, test and production roles with controlled promotion, scoped credentials and audit trails.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
Explanation
Correct Answer
D:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. In this case: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories.
Incorrect Answers
A:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. The decisive principle here is: Reproducibility requires more than saving an output weight file or project notebook.
B:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. The audit must corroborate: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
C:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment. In this case: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories.
Question 502
A risk-based audit engagement examines this operating condition: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- AI lifecycle controls include end-of-life risks; deleting an app icon is not an auditable retirement.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Reproducibility requires more than saving an output weight file or project notebook.
- AI development assurance starts from explicit risk-weighted requirements rather than retrospective claims of usefulness.
Explanation
Correct Answer
C:The corrective action is: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records. Independent audit evidence includes: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Incorrect Answers
A:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. The decisive principle here is: Reproducibility requires more than saving an output weight file or project notebook.
B:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment. For this situation: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories.
D:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. A proportionate response is: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
Question 503
An audit committee asks for a defensible assessment of this situation: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Review endpoint inventory, credentials, retained artifacts, disposal approvals, legal holds and restoration paths.
- Inspect supplier attestations, component inventory, signed digests, license constraints, SBOM and update notices.
- Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Explanation
Correct Answer
D:The corrective action is: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
Incorrect Answers
A:For this situation: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories.
B:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. For this situation: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories.
C:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. The decisive principle here is: Reproducibility requires more than saving an output weight file or project notebook.
Question 504
During an AI controls assessment, the team learns that a credit model's champion and challenger share a model identifier despite distinct training runs and validation histories. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Only approved traceable versions run and vulnerable components receive monitored remediation.
- Only signed approved model-data combinations reach production and privilege tests fail closed.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Approved model versions can be traced and materially reproduced with recorded dependencies.
Explanation
Correct Answer
D:This is an observable validation result for Reproducible model training and configuration management, not just management assurance: Approved model versions can be traced and materially reproduced with recorded dependencies.
Incorrect Answers
A:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. For this situation: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories.
B:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. For this situation: A credit model's champion and challenger share a model identifier despite distinct training runs and validation histories.
C:The auditor should instead examine: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Question 505
An independent AI auditor reviews this case: A loan model developer signs off their own fairness testing with no independent challenge or held-out dataset. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
- Use a decommission plan to disable endpoints, revoke access, retain required records and dispose of data appropriately.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
Explanation
Correct Answer
A:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. Independent audit evidence includes: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Incorrect Answers
B:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
C:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
D:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
Question 506
A governance and assurance review uncovers the following: A loan model developer signs off their own fairness testing with no independent challenge or held-out dataset. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Production AI monitoring must measure harmful outcomes, not merely infrastructure availability.
- A successful demo establishes feasibility, not sufficient reliability or authorization for real customer impact.
- Developer confidence cannot replace independent assessment for consequential model decisions.
Explanation
Correct Answer
D:The corrective action is: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance. This is the applicable AI audit principle for Independent validation before model deployment: Developer confidence cannot replace independent assessment for consequential model decisions.
Incorrect Answers
A:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. For this situation: A loan model developer signs off their own fairness testing with no independent challenge or held-out dataset.
B:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. The decisive principle here is: Developer confidence cannot replace independent assessment for consequential model decisions.
C:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
Question 507
A risk-based audit engagement examines this operating condition: A loan model developer signs off their own fairness testing with no independent challenge or held-out dataset. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
- Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
- Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
Explanation
Correct Answer
B:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. The corrective action is: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
Incorrect Answers
A:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. The decisive principle here is: Developer confidence cannot replace independent assessment for consequential model decisions.
C:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. For this situation: A loan model developer signs off their own fairness testing with no independent challenge or held-out dataset.
D:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. The auditor should instead examine: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Question 508
An audit committee asks for a defensible assessment of this situation: A loan model developer signs off their own fairness testing with no independent challenge or held-out dataset. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Each material requirement maps to a result and signoff before deploying the feature.
- An independent challenger confirms specified safety and performance requirements before go-live.
- Only approved traceable versions run and vulnerable components receive monitored remediation.
Explanation
Correct Answer
C:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. This is an observable validation result for Independent validation before model deployment, not just management assurance: An independent challenger confirms specified safety and performance requirements before go-live.
Incorrect Answers
A:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
B:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
D:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
Question 509
During an AI controls assessment, the team learns that a hospital outsources clinical model validation to the same supplier that sold the evaluation scores. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Separate development, test and production roles with controlled promotion, scoped credentials and audit trails.
- Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Define owner-led ongoing monitoring for quality, drift, safety, fairness and service impact with escalation.
Explanation
Correct Answer
B:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. The acceptable verified outcome is: An independent challenger confirms specified safety and performance requirements before go-live.
Incorrect Answers
A:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
C:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. Effective closure requires: An independent challenger confirms specified safety and performance requirements before go-live.
D:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. The audit must corroborate: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Question 510
An independent AI auditor reviews this case: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
- Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
Explanation
Correct Answers
B:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. This response fits the risk associated with Independent validation before model deployment: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
C:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. This evidence directly tests the asserted control for Independent validation before model deployment: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Incorrect Answers
A:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. For this situation: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores.
D:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
E:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. The audit must corroborate: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.