Isaca AAIA Practice Test 58
Question 570
An independent AI auditor reviews this case: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Use risk-aware canary populations, guardrail metrics, stop criteria and tested rollback or kill switch.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
- Inspect config state histories, privileges, break-glass logs, time bounds and post-incident review.
- Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
Explanation
Correct Answers
C:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
E:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. The corrective action is: Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
Incorrect Answers
A:This option concerns Canary releases and model rollback thresholds, which is a related but different AI governance concern. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
B:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
D:This option concerns AI configuration and emergency change traceability, which is a related but different AI governance concern. The decisive principle here is: AI changes can alter decisions without modifying the application UI or infrastructure code.
Question 571
A governance and assurance review uncovers the following: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Inspect prompt history, version diffs, tested attack cases, tool allowlists and release tickets.
- Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
- Compare provider release notes, model ID, output baselines, contractual notices and drift alerts.
Explanation
Correct Answer
C:For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
Incorrect Answers
A:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. Effective closure requires: Material behavioral changes receive proportionate approvals and do not bypass previously defined controls.
B:This option concerns Prompt template governance and injection-resilience changes, which is a related but different AI governance concern. In this case: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
D:This option concerns External foundation-model update change notices, which is a related but different AI governance concern. The decisive principle here is: AI changes can alter decisions without modifying the application UI or infrastructure code.
Question 572
A risk-based audit engagement examines this operating condition: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Authorized prompt versions perform as specified and fail closed on tested malicious context.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Material external changes are assessed before or promptly after adoption with controlled escalation.
- Material behavioral changes receive proportionate approvals and do not bypass previously defined controls.
Explanation
Correct Answer
D:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. In this case: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
Incorrect Answers
A:This option concerns Prompt template governance and injection-resilience changes, which is a related but different AI governance concern. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
B:It misses the key control principle: AI changes can alter decisions without modifying the application UI or infrastructure code.
C:This option concerns External foundation-model update change notices, which is a related but different AI governance concern. For this situation: An insurer silently swaps its risk score model and cannot explain changes in customer outcomes afterward.
Question 573
An audit committee asks for a defensible assessment of this situation: A recommendation system introduces new features that affect protected groups without triggering revalidation. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
- Separate development, test and production roles with controlled promotion, scoped credentials and audit trails.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Use risk-aware canary populations, guardrail metrics, stop criteria and tested rollback or kill switch.
Explanation
Correct Answer
A:The reason it is necessary is: AI changes can alter decisions without modifying the application UI or infrastructure code. In this case: A recommendation system introduces new features that affect protected groups without triggering revalidation.
Incorrect Answers
B:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. The decisive principle here is: AI changes can alter decisions without modifying the application UI or infrastructure code.
C:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. In this case: A recommendation system introduces new features that affect protected groups without triggering revalidation.
D:This option concerns Canary releases and model rollback thresholds, which is a related but different AI governance concern. In this case: A recommendation system introduces new features that affect protected groups without triggering revalidation.
Question 574
During an AI controls assessment, the team learns that a recommendation system introduces new features that affect protected groups without triggering revalidation. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- AI delivery involves models, data and code, so conventional app deployment approval alone may not cover behavior changes.
- Hosted AI services can change behavior outside local release cycles and require external change governance.
- AI changes can alter decisions without modifying the application UI or infrastructure code.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
Explanation
Correct Answer
C:This is the applicable AI audit principle for Model version change impact classification: AI changes can alter decisions without modifying the application UI or infrastructure code. The corrective action is: Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
Incorrect Answers
A:This option concerns AI data and model CI/CD separation, which is a related but different AI governance concern. For this situation: A recommendation system introduces new features that affect protected groups without triggering revalidation.
B:This option concerns External foundation-model update change notices, which is a related but different AI governance concern. In this case: A recommendation system introduces new features that affect protected groups without triggering revalidation.
D:This attractive shortcut does not resolve the Model version change impact classification weakness: Treat a model replacement as a routine security patch that cannot affect business outcomes. For this situation: A recommendation system introduces new features that affect protected groups without triggering revalidation.
Question 575
An independent AI auditor reviews this case: A recommendation system introduces new features that affect protected groups without triggering revalidation. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect source contracts, schema diffs, baseline distributions, feature tests and observed scoring changes.
- Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
- Inspect impact inventory, rollback record, retesting, business communications and lessons-learned owners.
Explanation
Correct Answer
B:This evidence directly tests the asserted control for Model version change impact classification: Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
Incorrect Answers
A:This option concerns AI dataset refresh and feature pipeline approvals, which is a related but different AI governance concern. In this case: A recommendation system introduces new features that affect protected groups without triggering revalidation.
C:For this situation: A recommendation system introduces new features that affect protected groups without triggering revalidation.
D:This option concerns AI model rollback and stakeholder communication, which is a related but different AI governance concern. It does not establish control effectiveness for Model version change impact classification.
Question 576
A governance and assurance review uncovers the following: A recommendation system introduces new features that affect protected groups without triggering revalidation. After management addresses the issue, which result MOST convincingly demonstrates effective remediation? Select ONE best answer.
- Material behavioral changes receive proportionate approvals and do not bypass previously defined controls.
- Material external changes are assessed before or promptly after adoption with controlled escalation.
- Emergency deviations expire or receive accountable renewal and critical controls are verified restored.
- Treat a model replacement as a routine security patch that cannot affect business outcomes.
Explanation
Correct Answer
A:This is an observable validation result for Model version change impact classification, not just management assurance: Material behavioral changes receive proportionate approvals and do not bypass previously defined controls.
Incorrect Answers
B:This option concerns External foundation-model update change notices, which is a related but different AI governance concern. For this situation: A recommendation system introduces new features that affect protected groups without triggering revalidation.
C:This option concerns AI configuration and emergency change traceability, which is a related but different AI governance concern. In this case: A recommendation system introduces new features that affect protected groups without triggering revalidation.
D:The auditor should instead examine: Compare before-after distributions, update artifacts, critical use cases, approval rules and rollback records.
Question 577
A risk-based audit engagement examines this operating condition: A customer-support assistant's system prompt is edited directly in production after one negative feedback report. What is the MOST appropriate risk-based audit recommendation that addresses the root control weakness? Select ONE best answer.
- Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing.
- Version prompt templates, permission boundaries and evaluation suites with staged approvals and rollback.
- Classify model, data, prompt and feature changes by potential behavior impact and require appropriate regression testing.
- Define calibrated thresholds and safe defer-to-human or rule-based fallbacks for uncertain or missing-data conditions.
Explanation
Correct Answer
B:The reason it is necessary is: Prompts are executable behavioral configuration and can change output and risk without code deployment. This is the proportionate control response to Prompt template governance and injection-resilience changes: Version prompt templates, permission boundaries and evaluation suites with staged approvals and rollback.
Incorrect Answers
A:This attractive shortcut does not resolve the Prompt template governance and injection-resilience changes weakness: Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing.
C:This option concerns Model version change impact classification, which is a related but different AI governance concern. It does not establish control effectiveness for Prompt template governance and injection-resilience changes.
D:This option concerns Threshold governance and safety fallbacks, which is a related but different AI governance concern. It does not establish control effectiveness for Prompt template governance and injection-resilience changes.
Question 578
An audit committee asks for a defensible assessment of this situation: A customer-support assistant's system prompt is edited directly in production after one negative feedback report. Which principle BEST explains why the observed practice is a material governance or control concern? Select ONE best answer.
- AI safety depends on configuration state as well as model artifacts; emergency bypasses need disciplined control.
- Prompts are executable behavioral configuration and can change output and risk without code deployment.
- Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing.
- Hosted AI services can change behavior outside local release cycles and require external change governance.
Explanation
Correct Answer
B:Independent audit evidence includes: Inspect prompt history, version diffs, tested attack cases, tool allowlists and release tickets. This is the applicable AI audit principle for Prompt template governance and injection-resilience changes: Prompts are executable behavioral configuration and can change output and risk without code deployment.
Incorrect Answers
A:This option concerns AI configuration and emergency change traceability, which is a related but different AI governance concern. It does not establish control effectiveness for Prompt template governance and injection-resilience changes.
C:This attractive shortcut does not resolve the Prompt template governance and injection-resilience changes weakness: Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing. For this situation: A customer-support assistant's system prompt is edited directly in production after one negative feedback report.
D:This option concerns External foundation-model update change notices, which is a related but different AI governance concern. It does not establish control effectiveness for Prompt template governance and injection-resilience changes.
Question 579
During an AI controls assessment, the team learns that a customer-support assistant's system prompt is edited directly in production after one negative feedback report. What evidence should the auditor examine FIRST to substantiate or challenge the control owner’s claim? Select ONE best answer.
- Inspect prompt history, version diffs, tested attack cases, tool allowlists and release tickets.
- Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing.
- Inspect canary assignment, segment KPIs, error thresholds, baseline model artifacts and rollback drill.
- Inspect tool-call traces, retrieval source trust, instruction hierarchy tests and denied action logs.
Explanation
Correct Answer
A:The reason it is necessary is: Prompts are executable behavioral configuration and can change output and risk without code deployment. This evidence directly tests the asserted control for Prompt template governance and injection-resilience changes: Inspect prompt history, version diffs, tested attack cases, tool allowlists and release tickets.
Incorrect Answers
B:This attractive shortcut does not resolve the Prompt template governance and injection-resilience changes weakness: Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing. The auditor should instead examine: Inspect prompt history, version diffs, tested attack cases, tool allowlists and release tickets.
C:This option concerns Canary releases and model rollback thresholds, which is a related but different AI governance concern. It does not establish control effectiveness for Prompt template governance and injection-resilience changes.
D:This option concerns Indirect prompt injection through retrieved content, which is a related but different AI governance concern. It does not establish control effectiveness for Prompt template governance and injection-resilience changes.
Question 580
An independent AI auditor reviews this case: A customer-support assistant's system prompt is edited directly in production after one negative feedback report. Which TWO complementary elements should the auditor seek: an effective corrective control and the most probative independent evidence? Select TWO answers.
- Inspect calibration, confusion matrices, missing-input behavior, escalation queue and approval history.
- Version prompt templates, permission boundaries and evaluation suites with staged approvals and rollback.
- Log emergency AI configuration changes, require proportionate approval, expiry and post-change validation.
- Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing.
- Inspect prompt history, version diffs, tested attack cases, tool allowlists and release tickets.
Explanation
Correct Answers
B:The reason it is necessary is: Prompts are executable behavioral configuration and can change output and risk without code deployment. This response fits the risk associated with Prompt template governance and injection-resilience changes: Version prompt templates, permission boundaries and evaluation suites with staged approvals and rollback.
E:The reason it is necessary is: Prompts are executable behavioral configuration and can change output and risk without code deployment. For this situation: A customer-support assistant's system prompt is edited directly in production after one negative feedback report.
Incorrect Answers
A:This option concerns Threshold governance and safety fallbacks, which is a related but different AI governance concern. The decisive principle here is: Prompts are executable behavioral configuration and can change output and risk without code deployment.
C:This option concerns AI configuration and emergency change traceability, which is a related but different AI governance concern. The decisive principle here is: Prompts are executable behavioral configuration and can change output and risk without code deployment.
D:This attractive shortcut does not resolve the Prompt template governance and injection-resilience changes weakness: Allow any content editor to hot-swap production prompts and regard that as nontechnical copy editing. Effective closure requires: Authorized prompt versions perform as specified and fail closed on tested malicious context.
