Isaca AAIA Practice Test 57
Question 560
An independent AI auditor reviews this case: A pilot fraud model receives the same global authorizations as a mature production decision service. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Version model artifacts, code, environments, hyperparameters and data references with reproducible promotion records.
- Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
- Review reviewer independence, held-out data provenance, challenge results, exceptions and signoffs.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
- Promote a prototype directly because it impressed stakeholders in a small curated demonstration.
Explanation
Correct Answers
B:The reason it is necessary is: A successful demo establishes feasibility, not sufficient reliability or authorization for real customer impact. This response fits the risk associated with AI prototype experimentation versus production safety gates: Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
D:The reason it is necessary is: A successful demo establishes feasibility, not sufficient reliability or authorization for real customer impact. The acceptable verified outcome is: Production release occurs only after required controls and independent signoffs are demonstrated.
Incorrect Answers
A:This option concerns Reproducible model training and configuration management, which is a related but different AI governance concern. For this situation: A pilot fraud model receives the same global authorizations as a mature production decision service.
C:This option concerns Independent validation before model deployment, which is a related but different AI governance concern. For this situation: A pilot fraud model receives the same global authorizations as a mature production decision service.
E:The auditor should instead examine: Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
Question 561
A governance and assurance review uncovers the following: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Create traceable business, human impact, security and quality requirements with measurable release and acceptance gates.
- Promote a prototype directly because it impressed stakeholders in a small curated demonstration.
- Inventory and authenticate model components, third-party datasets and software dependencies throughout acquisition and updates.
- Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
Explanation
Correct Answer
D:The reason it is necessary is: A successful demo establishes feasibility, not sufficient reliability or authorization for real customer impact. In this case: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed.
Incorrect Answers
A:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. A proportionate response is: Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
B:This attractive shortcut does not resolve the AI prototype experimentation versus production safety gates weakness: Promote a prototype directly because it impressed stakeholders in a small curated demonstration. In this case: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed.
C:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. The audit must corroborate: Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
Question 562
A risk-based audit engagement examines this operating condition: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- A successful demo establishes feasibility, not sufficient reliability or authorization for real customer impact.
- AI development assurance starts from explicit risk-weighted requirements rather than retrospective claims of usefulness.
- Promote a prototype directly because it impressed stakeholders in a small curated demonstration.
- AI lifecycle controls include end-of-life risks; deleting an app icon is not an auditable retirement.
Explanation
Correct Answer
A:Independent audit evidence includes: Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records. The acceptable verified outcome is: Production release occurs only after required controls and independent signoffs are demonstrated.
Incorrect Answers
B:This option concerns AI requirements-to-acceptance traceability, which is a related but different AI governance concern. For this situation: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed.
C:This attractive shortcut does not resolve the AI prototype experimentation versus production safety gates weakness: Promote a prototype directly because it impressed stakeholders in a small curated demonstration. For this situation: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed.
D:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. It does not establish control effectiveness for AI prototype experimentation versus production safety gates.
Question 563
An audit committee asks for a defensible assessment of this situation: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Promote a prototype directly because it impressed stakeholders in a small curated demonstration.
- Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
- Inspect pipeline roles, artifact signatures, dataset permissions, approval gates and production change logs.
- Review endpoint inventory, credentials, retained artifacts, disposal approvals, legal holds and restoration paths.
Explanation
Correct Answer
B:This evidence directly tests the asserted control for AI prototype experimentation versus production safety gates: Inspect sandbox isolation, tool permissions, go-live checklist, safety test results and deployment records.
Incorrect Answers
A:For this situation: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed.
C:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. A proportionate response is: Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
D:This option concerns Controlled model retirement and data disposition, which is a related but different AI governance concern. The decisive principle here is: A successful demo establishes feasibility, not sufficient reliability or authorization for real customer impact.
Question 564
During an AI controls assessment, the team learns that a chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Only approved traceable versions run and vulnerable components receive monitored remediation.
- Promote a prototype directly because it impressed stakeholders in a small curated demonstration.
- Production release occurs only after required controls and independent signoffs are demonstrated.
- Only signed approved model-data combinations reach production and privilege tests fail closed.
Explanation
Correct Answer
C:This is an observable validation result for AI prototype experimentation versus production safety gates, not just management assurance: Production release occurs only after required controls and independent signoffs are demonstrated.
Incorrect Answers
A:This option concerns AI vendor component and supply chain lifecycle, which is a related but different AI governance concern. For this situation: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed.
B:Separate sandbox experimentation from production through constrained access, risk approval and evidence-based readiness criteria.
D:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. For this situation: A chatbot demo's disclaimer is removed before launch although unsafe response tests were never completed.
Question 565
An independent AI auditor reviews this case: A model vendor changes a major release but the client records only an ordinary infrastructure patch ticket. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Require contractual update notifications, version observability, revalidation and compensating fallback for material supplier changes.
- Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Use risk-aware canary populations, guardrail metrics, stop criteria and tested rollback or kill switch.
Explanation
Correct Answer
B:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. Independent audit evidence includes: Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
Incorrect Answers
A:This option concerns External foundation-model update change notices, which is a related but different AI governance concern. It does not establish control effectiveness for Model version change impact classification.
C:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes.
D:This option concerns Canary releases and model rollback thresholds, which is a related but different AI governance concern. It does not establish control effectiveness for Model version change impact classification.
Question 566
A governance and assurance review uncovers the following: A model vendor changes a major release but the client records only an ordinary infrastructure patch ticket. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- Prompts are executable behavioral configuration and can change output and risk without code deployment.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- AI safety depends on configuration state as well as model artifacts; emergency bypasses need disciplined control.
- AI changes can alter decisions without modifying the application UI or infrastructure code.
Explanation
Correct Answer
D:This is the applicable AI audit principle for Model version change impact classification: AI changes can alter decisions without modifying the application UI or infrastructure code. Independent audit evidence includes: Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
Incorrect Answers
A:This option concerns Prompt template governance and injection-resilience changes, which is a related but different AI governance concern. It does not establish control effectiveness for Model version change impact classification.
B:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. For this situation: A model vendor changes a major release but the client records only an ordinary infrastructure patch ticket.
C:This option concerns AI configuration and emergency change traceability, which is a related but different AI governance concern. It does not establish control effectiveness for Model version change impact classification.
Question 567
A risk-based audit engagement examines this operating condition: A model vendor changes a major release but the client records only an ordinary infrastructure patch ticket. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect source contracts, schema diffs, baseline distributions, feature tests and observed scoring changes.
- Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Inspect prompt history, version diffs, tested attack cases, tool allowlists and release tickets.
Explanation
Correct Answer
B:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. This evidence directly tests the asserted control for Model version change impact classification: Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
Incorrect Answers
A:This option concerns AI dataset refresh and feature pipeline approvals, which is a related but different AI governance concern. It does not establish control effectiveness for Model version change impact classification.
C:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. The auditor should instead examine: Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
D:This option concerns Prompt template governance and injection-resilience changes, which is a related but different AI governance concern. For this situation: A model vendor changes a major release but the client records only an ordinary infrastructure patch ticket.
Question 568
An audit committee asks for a defensible assessment of this situation: A model vendor changes a major release but the client records only an ordinary infrastructure patch ticket. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Only signed approved model-data combinations reach production and privilege tests fail closed.
- Material behavioral changes receive proportionate approvals and do not bypass previously defined controls.
- Authorized prompt versions perform as specified and fail closed on tested malicious context.
Explanation
Correct Answer
C:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. This is an observable validation result for Model version change impact classification, not just management assurance: Material behavioral changes receive proportionate approvals and do not bypass previously defined controls.
Incorrect Answers
A:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
B:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. It does not establish control effectiveness for Model version change impact classification.
D:This option concerns Prompt template governance and injection-resilience changes, which is a related but different AI governance concern. The decisive principle here is: AI changes can alter decisions without modifying the application UI or infrastructure code.
Question 569
During an AI controls assessment, the team learns that an insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Use risk-aware canary populations, guardrail metrics, stop criteria and tested rollback or kill switch.
- Control schema, source, feature and data refreshes using integrity and semantic compatibility tests.
- Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
Explanation
Correct Answer
D:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. The acceptable verified outcome is: Material behavioral changes receive proportionate approvals and do not bypass previously defined controls.
Incorrect Answers
A:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. In this case: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
B:This option concerns Canary releases and model rollback thresholds, which is a related but different AI governance concern. The decisive principle here is: AI changes can alter decisions without modifying the application UI or infrastructure code.
C:This option concerns AI dataset refresh and feature pipeline approvals, which is a related but different AI governance concern. The decisive principle here is: AI changes can alter decisions without modifying the application UI or infrastructure code.
Question 570
An independent AI auditor reviews this case: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Use risk-aware canary populations, guardrail metrics, stop criteria and tested rollback or kill switch.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
- Inspect config state histories, privileges, break-glass logs, time bounds and post-incident review.
- Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
Explanation
Correct Answers
C:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
E:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. The corrective action is: Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
Incorrect Answers
A:This option concerns Canary releases and model rollback thresholds, which is a related but different AI governance concern. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
B:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
D:This option concerns AI configuration and emergency change traceability, which is a related but different AI governance concern. The decisive principle here is: AI changes can alter decisions without modifying the application UI or infrastructure code.
