Isaca AAIA Practice Test 52
Question 510
An independent AI auditor reviews this case: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
- Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
Explanation
Correct Answers
B:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. This response fits the risk associated with Independent validation before model deployment: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
C:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. This evidence directly tests the asserted control for Independent validation before model deployment: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Incorrect Answers
A:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. For this situation: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores.
D:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. It does not establish control effectiveness for Independent validation before model deployment.
E:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. The audit must corroborate: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Question 511
A governance and assurance review uncovers the following: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
- Inspect live KPIs and KRIs, baselines, incident triggers, ownership and corrective decisions.
- Inspect signed use cases, requirement trace matrix, outcome thresholds, known limitations and test cases.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
Explanation
Correct Answer
A:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. For this situation: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores.
Incorrect Answers
B:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. For this situation: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores.
C:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. The decisive principle here is: Developer confidence cannot replace independent assessment for consequential model decisions.
D:It misses the key control principle: Developer confidence cannot replace independent assessment for consequential model decisions.
Question 512
A risk-based audit engagement examines this operating condition: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Approved model versions can be traced and materially reproduced with recorded dependencies.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Threshold breaches lead to timely investigation and documented changes to protect affected users.
- An independent challenger confirms specified safety and performance requirements before go-live.
Explanation
Correct Answer
D:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. In this case: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores.
Incorrect Answers
A:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. The decisive principle here is: Developer confidence cannot replace independent assessment for consequential model decisions.
B:For this situation: A hospital outsources clinical model validation to the same supplier that sold the evaluation scores.
C:This option concerns AI post-deployment monitoring and KRI ownership, which is a related but different AI governance concern. A proportionate response is: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
Question 513
An audit committee asks for a defensible assessment of this situation: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
Explanation
Correct Answer
A:The reason it is necessary is: Developer confidence cannot replace independent assessment for consequential model decisions. In this case: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred.
Incorrect Answers
B:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. The audit must corroborate: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
C:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. A proportionate response is: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
D:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. In this case: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred.
Question 514
During an AI controls assessment, the team learns that an energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Developer confidence cannot replace independent assessment for consequential model decisions.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Reproducibility requires more than saving an output weight file or project notebook.
- AI development assurance starts from explicit risk-weighted requirements rather than retrospective claims of usefulness.
Explanation
Correct Answer
A:The corrective action is: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance. Independent audit evidence includes: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Incorrect Answers
B:This attractive shortcut does not resolve the Independent validation before model deployment weakness: Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment. For this situation: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred.
C:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. For this situation: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred.
D:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. The audit must corroborate: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Question 515
An independent AI auditor reviews this case: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
- Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
- Review endpoint inventory, credentials, retained artifacts, disposal approvals, legal holds and restoration paths.
Explanation
Correct Answer
C:The corrective action is: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
Incorrect Answers
A:For this situation: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred.
B:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. A proportionate response is: Require risk-proportionate independent model validation, controlled challenge samples and release acceptance.
D:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. The decisive principle here is: Developer confidence cannot replace independent assessment for consequential model decisions.
Question 516
A governance and assurance review uncovers the following: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Each material requirement maps to a result and signoff before deploying the feature.
- The retired model cannot serve unauthorized predictions and remaining data is governed by approved retention.
- Mark self-authored training accuracy as sufficient independent validation of a high-impact deployment.
- An independent challenger confirms specified safety and performance requirements before go-live.
Explanation
Correct Answer
D:This is an observable validation result for Independent validation before model deployment, not just management assurance: An independent challenger confirms specified safety and performance requirements before go-live.
Incorrect Answers
A:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. For this situation: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred.
B:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. For this situation: An energy utility deploys an outage predictor after one internal demo although no independent operational scenario tests occurred.
C:The auditor should instead examine: Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
Question 517
A risk-based audit engagement examines this operating condition: An AI development team gives a shared service account authority to change training datasets and promote production weights. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Separate development, test and production roles with controlled promotion, scoped credentials and audit trails.
- Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
- Allow every data scientist to alter training inputs and production model endpoints without change approval.
- Inventory and authenticate model components, third-party datasets and software dependencies throughout acquisition and updates.
Explanation
Correct Answer
A:The reason it is necessary is: AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes. The acceptable verified outcome is: Only signed approved model-data combinations reach production and privilege tests fail closed.
Incorrect Answers
B:This option concerns AI prototype experimentation versus production safety gates, which is a related but different AI governance concern. It does not establish control effectiveness for AI data and model CI/CD separation.
C:This attractive shortcut does not resolve the AI data and model CI/CD separation weakness: Allow every data scientist to alter training inputs and production model endpoints without change approval.
D:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. It does not establish control effectiveness for AI data and model CI/CD separation.
Question 518
An audit committee asks for a defensible assessment of this situation: An AI development team gives a shared service account authority to change training datasets and promote production weights. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- An AI incident ends only after decision impacts, service restoration and recurrence risk are addressed.
- Allow every data scientist to alter training inputs and production model endpoints without change approval.
- Reproducibility requires more than saving an output weight file or project notebook.
- AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes.
Explanation
Correct Answer
D:The acceptable verified outcome is: Only signed approved model-data combinations reach production and privilege tests fail closed. Independent audit evidence includes: Inspect pipeline roles, artifact signatures, dataset permissions, approval gates and production change logs.
Incorrect Answers
A:This option concerns AI model rollback and stakeholder communication, which is a related but different AI governance concern. It does not establish control effectiveness for AI data and model CI/CD separation.
B:This attractive shortcut does not resolve the AI data and model CI/CD separation weakness: Allow every data scientist to alter training inputs and production model endpoints without change approval. For this situation: An AI development team gives a shared service account authority to change training datasets and promote production weights.
C:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. It does not establish control effectiveness for AI data and model CI/CD separation.
Question 519
During an AI controls assessment, the team learns that an AI development team gives a shared service account authority to change training datasets and promote production weights. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
- Inspect pipeline roles, artifact signatures, dataset permissions, approval gates and production change logs.
- Allow every data scientist to alter training inputs and production model endpoints without change approval.
- Inspect image dependencies, seeds, pipelines, registry signatures, lineage and controlled reruns.
Explanation
Correct Answer
B:The reason it is necessary is: AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes. This evidence directly tests the asserted control for AI data and model CI/CD separation: Inspect pipeline roles, artifact signatures, dataset permissions, approval gates and production change logs.
Incorrect Answers
A:This option concerns Independent validation before model deployment, which is a related but different AI governance concern. It does not establish control effectiveness for AI data and model CI/CD separation.
C:This attractive shortcut does not resolve the AI data and model CI/CD separation weakness: Allow every data scientist to alter training inputs and production model endpoints without change approval. The auditor should instead examine: Inspect pipeline roles, artifact signatures, dataset permissions, approval gates and production change logs.
D:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. The decisive principle here is: AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes.
Question 520
An independent AI auditor reviews this case: An AI development team gives a shared service account authority to change training datasets and promote production weights. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Allow every data scientist to alter training inputs and production model endpoints without change approval.
- Inspect signed use cases, requirement trace matrix, outcome thresholds, known limitations and test cases.
- Separate development, test and production roles with controlled promotion, scoped credentials and audit trails.
- Inventory and authenticate model components, third-party datasets and software dependencies throughout acquisition and updates.
- Inspect pipeline roles, artifact signatures, dataset permissions, approval gates and production change logs.
Explanation
Correct Answers
C:The reason it is necessary is: AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes. This response fits the risk associated with AI data and model CI/CD separation: Separate development, test and production roles with controlled promotion, scoped credentials and audit trails.
E:The reason it is necessary is: AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes. For this situation: An AI development team gives a shared service account authority to change training datasets and promote production weights.
Incorrect Answers
A:This attractive shortcut does not resolve the AI data and model CI/CD separation weakness: Allow every data scientist to alter training inputs and production model endpoints without change approval. Effective closure requires: Only signed approved model-data combinations reach production and privilege tests fail closed.
B:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. It does not establish control effectiveness for AI data and model CI/CD separation.
D:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. The decisive principle here is: AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes.
