Isaca AAIA Practice Test 50
Question 490
An independent AI auditor reviews this case: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect config state histories, privileges, break-glass logs, time bounds and post-incident review.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Inspect signed use cases, requirement trace matrix, outcome thresholds, known limitations and test cases.
- Create traceable business, human impact, security and quality requirements with measurable release and acceptance gates.
- Treat a successful prototype demonstration as complete evidence of all unspecified requirements.
Explanation
Correct Answers
C:The acceptable verified outcome is: Each material requirement maps to a result and signoff before deploying the feature.
D:The reason it is necessary is: AI development assurance starts from explicit risk-weighted requirements rather than retrospective claims of usefulness. For this situation: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments.
Incorrect Answers
A:This option concerns AI configuration and emergency change traceability, which is a related but different AI governance concern. The decisive principle here is: AI development assurance starts from explicit risk-weighted requirements rather than retrospective claims of usefulness.
B:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. For this situation: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments.
E:This attractive shortcut does not resolve the AI requirements-to-acceptance traceability weakness: Treat a successful prototype demonstration as complete evidence of all unspecified requirements. For this situation: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments.
Question 491
A governance and assurance review uncovers the following: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect signed use cases, requirement trace matrix, outcome thresholds, known limitations and test cases.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Treat a successful prototype demonstration as complete evidence of all unspecified requirements.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
Explanation
Correct Answer
A:For this situation: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments.
Incorrect Answers
B:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. It does not establish control effectiveness for AI requirements-to-acceptance traceability.
C:The auditor should instead examine: Inspect signed use cases, requirement trace matrix, outcome thresholds, known limitations and test cases.
D:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. The decisive principle here is: AI development assurance starts from explicit risk-weighted requirements rather than retrospective claims of usefulness.
Question 492
A risk-based audit engagement examines this operating condition: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- An independent rerun can identify the exact source data and material transformation decisions for a sampled model version.
- The retired model cannot serve unauthorized predictions and remaining data is governed by approved retention.
- Each material requirement maps to a result and signoff before deploying the feature.
- Treat a successful prototype demonstration as complete evidence of all unspecified requirements.
Explanation
Correct Answer
C:This is an observable validation result for AI requirements-to-acceptance traceability, not just management assurance: Each material requirement maps to a result and signoff before deploying the feature.
Incorrect Answers
A:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. For this situation: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments.
B:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. In this case: A fraud project lists model accuracy as its only requirement even though it will automatically hold customer payments.
D:Effective closure requires: Each material requirement maps to a result and signoff before deploying the feature.
Question 493
An audit committee asks for a defensible assessment of this situation: A team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Inventory and authenticate model components, third-party datasets and software dependencies throughout acquisition and updates.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Create traceable business, human impact, security and quality requirements with measurable release and acceptance gates.
Explanation
Correct Answer
A:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. This is the proportionate control response to Reproducible model training and configuration management: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
Incorrect Answers
B:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
C:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment.
D:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
Question 494
During an AI controls assessment, the team learns that a team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- AI supply-chain risk includes pre-trained weights, code, data sources and hosted capabilities.
- Reproducibility requires more than saving an output weight file or project notebook.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Developer confidence cannot replace independent assessment for consequential model decisions.
Explanation
Correct Answer
B:The corrective action is: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records. This is the applicable AI audit principle for Reproducible model training and configuration management: Reproducibility requires more than saving an output weight file or project notebook.
Incorrect Answers
A:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. For this situation: A team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved.
C:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment. For this situation: A team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved.
D:This option concerns Independent validation before model deployment, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
Question 495
An independent AI auditor reviews this case: A team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
- Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Inspect pipeline roles, artifact signatures, dataset permissions, approval gates and production change logs.
Explanation
Correct Answer
B:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. The corrective action is: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
Incorrect Answers
A:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
C:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment. The auditor should instead examine: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
D:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
Question 496
A governance and assurance review uncovers the following: A team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- An independent challenger confirms specified safety and performance requirements before go-live.
- Approved model versions can be traced and materially reproduced with recorded dependencies.
- Each material requirement maps to a result and signoff before deploying the feature.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
Explanation
Correct Answer
B:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. This is an observable validation result for Reproducible model training and configuration management, not just management assurance: Approved model versions can be traced and materially reproduced with recorded dependencies.
Incorrect Answers
A:This option concerns Independent validation before model deployment, which is a related but different AI governance concern. For this situation: A team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved.
C:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. For this situation: A team cannot reproduce a model release because random seeds, libraries and hyperparameters are not preserved.
D:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment. Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
Question 497
A risk-based audit engagement examines this operating condition: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Separate development, test and production roles with controlled promotion, scoped credentials and audit trails.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Create traceable business, human impact, security and quality requirements with measurable release and acceptance gates.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
Explanation
Correct Answer
D:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. The acceptable verified outcome is: Approved model versions can be traced and materially reproduced with recorded dependencies.
Incorrect Answers
A:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. The decisive principle here is: Reproducibility requires more than saving an output weight file or project notebook.
B:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment. Effective closure requires: Approved model versions can be traced and materially reproduced with recorded dependencies.
C:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. The decisive principle here is: Reproducibility requires more than saving an output weight file or project notebook.
Question 498
An audit committee asks for a defensible assessment of this situation: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Reproducibility requires more than saving an output weight file or project notebook.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Developer confidence cannot replace independent assessment for consequential model decisions.
- AI development assurance starts from explicit risk-weighted requirements rather than retrospective claims of usefulness.
Explanation
Correct Answer
A:The corrective action is: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records. The acceptable verified outcome is: Approved model versions can be traced and materially reproduced with recorded dependencies.
Incorrect Answers
B:This attractive shortcut does not resolve the Reproducible model training and configuration management weakness: Assume a Git commit for application code uniquely identifies the model, training data and build environment. For this situation: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version.
C:This option concerns Independent validation before model deployment, which is a related but different AI governance concern. The decisive principle here is: Reproducibility requires more than saving an output weight file or project notebook.
D:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. For this situation: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version.
Question 499
During an AI controls assessment, the team learns that an engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
- Inspect dataset snapshots, feature pipeline commits, record counts, model registry links and training execution manifests.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
Explanation
Correct Answer
A:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. This evidence directly tests the asserted control for Reproducible model training and configuration management: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Incorrect Answers
B:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. The decisive principle here is: Reproducibility requires more than saving an output weight file or project notebook.
C:This option concerns AI training dataset lineage and reproducibility, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
D:It misses the key control principle: Reproducibility requires more than saving an output weight file or project notebook.
Question 500
An independent AI auditor reviews this case: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version. Which THREE elements together provide a defensible AI audit conclusion: the corrective control, the evidence to test it, and a verified outcome? Select THREE answers.
- Assume a Git commit for application code uniquely identifies the model, training data and build environment.
- Review endpoint inventory, credentials, retained artifacts, disposal approvals, legal holds and restoration paths.
- Approved model versions can be traced and materially reproduced with recorded dependencies.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Create traceable business, human impact, security and quality requirements with measurable release and acceptance gates.
- Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Explanation
Correct Answers
C:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. Independent audit evidence includes: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
D:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. This response fits the risk associated with Reproducible model training and configuration management: Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
F:The reason it is necessary is: Reproducibility requires more than saving an output weight file or project notebook. For this situation: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version.
Incorrect Answers
A:For this situation: An engineering group fine-tunes an LLM with changing data files and writes the resulting weights directly over a production version.
B:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. It does not establish control effectiveness for Reproducible model training and configuration management.
E:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. The audit must corroborate: Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
