All ISTQB CT-AI certification exam dumps, study guide, training courses are Prepared by industry experts. PrepAway's ETE files povide the CT-AI ISTQB Certified Tester - AI Testing practice test questions and answers & exam dumps, study guide and training courses help you study and pass hassle-free!

ISTQB CT-AI v2.0: Testing AI and Machine Learning Systems

ISTQB Certified Tester AI Testing (CT-AI) has moved to version 2.0. The current syllabus focuses on testing AI-based systems, including machine-learning systems and generative-AI systems such as large language models. It replaces CT-AI v1.0, whose English exams and training remain available only until April 21, 2027, with non-English v1.0 availability ending October 21, 2027. Current candidates should therefore build around v2.0 rather than treating the earlier syllabus as the default.

CT-AI is about testing systems that contain AI, not primarily about using AI to help a tester do ordinary testing work. That second use case is now addressed by the CT-GenAI certification. Both sit inside the ISTQB certification scheme and require Foundation Level, but they solve different quality-engineering problems.

AI-based systems behave differently from deterministic business logic

Traditional software often maps defined inputs to predictable outputs. Machine-learning components may produce probabilistic results, shift as data changes, or behave differently around inputs that are semantically similar but statistically unusual. The tester therefore needs a quality model that recognizes uncertainty instead of assuming every case has one exact expected value.

That does not make AI untestable. It changes the evidence. Testers can evaluate data properties, model performance, robustness, fairness, safety, explainability, and behavior across representative and adversarial conditions. The key is to define what acceptable behavior means for the system and the risks it creates.

Data quality is part of the test basis

Machine-learning behavior is strongly influenced by the data used to train, validate, and test models. Missing values, label errors, class imbalance, leakage, stale distributions, duplicate records, and biased sampling can undermine a model before algorithm selection becomes the main concern. The broader discipline of data quality is therefore directly relevant to AI testing.

Testers need to understand lineage and separation between datasets. Training data teaches the model; validation data supports model selection or tuning; test data should provide an independent basis for estimating behavior. If the same information leaks across those boundaries, reported performance can look better than the model will achieve in real use.

Model metrics need to be interpreted in context

CT-AI expects candidates to work with machine-learning performance concepts rather than treat a single accuracy number as definitive. Classification metrics can include precision, recall, specificity, and related measures; the appropriate measure depends on the cost of different mistakes. A primer on machine learning helps explain why model evaluation must be tied to the problem rather than to a universal score.

A medical-screening model, fraud detector, recommendation engine, and content classifier may all need different error trade-offs. Testers should ask which false positives and false negatives matter, how performance varies across groups or conditions, and whether the metric remains stable when the input distribution changes.

Neural networks add opacity and sensitivity that tests must address

Neural networks can learn complex relationships without exposing a simple rule set that a tester can inspect line by line. This makes black-box behavior, input transformations, activation patterns, robustness checks, and model-level metrics more important. Small input changes can sometimes produce unexpectedly large output differences, especially near decision boundaries.

Testing should therefore include representative data, boundary-like cases in the feature space, perturbations, invalid inputs, and scenarios that challenge assumptions. The objective is not to prove that the network is mathematically perfect; it is to discover where behavior becomes unreliable for the intended use.

AI quality includes fairness, transparency, safety, and robustness

AI systems can create harm even when their average performance is strong. Biased outcomes, opaque decisions, unsafe recommendations, brittle behavior, or inappropriate autonomy can make a model unacceptable. Fair AI concepts are relevant because testers may need to compare performance across meaningful groups and identify where data or model behavior creates systematic disadvantage.

Transparency and explainability also affect testability. Some systems need to provide reasons or supporting evidence that users, auditors, or operators can understand. Testers can evaluate whether explanations are available, consistent, useful, and aligned with the actual model behavior rather than merely plausible-sounding narratives.

Generative AI and large language models require specialized test oracles

Large language models may produce multiple acceptable outputs for the same prompt, which makes exact string comparison a weak oracle. Testers need criteria for relevance, factuality, safety, consistency, format, instruction following, and domain-specific correctness. Understanding the distinction between generative AI and large language models helps teams choose the right test scope.

Evaluation may combine human review, reference answers, rubric scoring, automated metrics, adversarial prompts, and metamorphic relationships. Because generated output can be fluent while wrong, evaluation must separate presentation quality from factual or functional quality.

AI testing should follow the machine-learning lifecycle

Version 2.0 uses a lifecycle-oriented view that includes input-data testing, model testing, and machine-learning development testing. This helps testers place evidence at the stage where a problem can be detected most effectively. Data-schema issues belong early; model-performance and robustness checks belong around training and evaluation; system-level tests examine how the AI component behaves inside the larger product.

Lifecycle thinking also improves traceability. A quality problem may originate in data collection, preprocessing, labeling, feature engineering, model selection, prompt design, integration, or monitoring. Tests should help isolate the source rather than report every symptom simply as “the AI was wrong.”

Operational monitoring matters because AI behavior can drift

A model that performs well during release testing can degrade as users, data, language, products, or external conditions change. AI quality therefore extends into monitoring and governance. The ideas behind AI trust, risk, and security management are relevant because organizations need triggers for investigation, retraining, rollback, or human intervention when behavior moves outside acceptable bounds.

Testers can contribute by defining baseline distributions, quality thresholds, canary scenarios, drift indicators, and regression datasets. Production evidence then feeds future testing rather than remaining isolated in operations dashboards.

The current CT-AI v2.0 exam reflects the new scope

ISTQB currently lists CT-AI v2.0 at 40 questions, 44 possible points, a passing score of 29, and a standard duration of 60 minutes. The content covers AI foundations, AI-specific quality characteristics, machine learning, input-data testing, model testing, machine-learning development testing, and testing generative AI and large language models.

Candidates must hold CTFL. Anyone studying older v1.0 material should compare it against v2.0 carefully because the scope has been reorganized and the use of AI to support general testing has moved toward CT-GenAI. Version awareness is therefore part of preparation, not an administrative detail.

Adversarial testing is particularly important for AI systems because normal validation data may not expose how the model behaves when inputs are intentionally crafted to confuse, manipulate, or bypass it. The exact techniques depend on the system, but the principle is to challenge assumptions about input distribution, prompt behavior, robustness, and abuse. Red-team exercises can complement conventional tests by exploring harmful or unexpected outcomes that product teams did not design as normal use cases.

Metamorphic testing is useful when an exact oracle is difficult. A tester identifies relationships that should hold across transformed inputs. For example, a small irrelevant change may be expected not to reverse a classification, or equivalent representations may be expected to produce materially consistent outcomes. Violating the relationship can reveal instability even when there is no single authoritative answer for the original input.

Human oversight is part of quality design for AI systems used in consequential decisions. Testing should verify not only model outputs but also escalation, override, explanation, review, and fallback mechanisms. A system can have strong model metrics and still be unsafe if operators cannot recognize uncertainty or intervene when the model is outside its intended operating conditions.

Integration testing matters because the model is only one component. Preprocessing code, feature stores, retrieval systems, prompt templates, business rules, APIs, caching, post-processing, and user interfaces can change the final behavior. A model may be correct in isolation while the product fails because labels are mapped incorrectly or context is assembled badly. AI testing therefore needs system-level evidence in addition to model-level metrics.

Governance artifacts can also become test inputs. Model cards, data documentation, risk assessments, intended-use statements, prohibited uses, and monitoring plans describe how the organization expects the AI system to behave. Testers can review whether those claims are specific, measurable, and reflected in actual test coverage. This connects technical evidence with the accountability needed for responsible deployment.

For v2.0 preparation, hands-on familiarity with small machine-learning workflows is useful because the syllabus emphasizes practical understanding. Candidates should be comfortable reasoning about datasets, model performance, quality characteristics, and lifecycle testing rather than learning AI terminology in isolation. The exam remains a testing qualification: every AI concept ultimately matters because it changes how quality evidence should be designed or interpreted.

Test coverage for AI systems should include known limits as well as expected use. If a model is approved only for certain languages, document types, age groups, geographies, or confidence ranges, tests should verify what happens outside those boundaries. The safest outcome may be refusal, escalation, or a warning rather than a low-confidence answer presented as authoritative. Operating limits are therefore part of the testable product contract.

AI regression testing also needs version discipline. A model update, prompt change, retrieval source, preprocessing change, or safety-policy adjustment can alter behavior even when application code is unchanged. Teams should preserve representative regression datasets and high-risk examples so they can compare releases consistently and decide whether improvements in one area caused deterioration elsewhere.

The same discipline applies to AI safety filters and guardrails. A model may be configured to refuse unsafe requests, redact sensitive content, or route low-confidence cases to a human. Those controls need positive and negative testing of their own. Testers should verify that legitimate use is not blocked unnecessarily while harmful or prohibited behavior is constrained consistently across relevant languages, formats, and interaction patterns.

ISTQB CT-AI practice test questions and answers, training course, study guide are uploaded in ETE Files format by real users. Study and Pass CT-AI ISTQB Certified Tester - AI Testing certification exam dumps & practice test questions and answers are to help students.