ISTQB CT-AI - ISTQBCertified Tester AI Testing Exam

In the syllabus, theevolutioncharacteristic for AI-based systems means the ability of the system to evolve and adapt its behavior in response to changes in the environment or in its own performance:

"Evolution is the system’s ability to change itself to adapt to new situations, different hardware, or a changing operational environment."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.3)

The syllabus describes exploratory testing as especially useful when there are poor specifications or test oracle problems:

“Exploratory testing is especially useful when there are poor specifications or test oracle problems, which is often the case for AI-based systems.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.6, page 70 of 99)

The syllabus discusses using AI-based tools to reduce GUI test brittleness:

"AI can be used to reduce the brittleness of this approach, by employing AI-based tools to identify the correct objects using various criteria (e.g., XPath, label, id, class, X/Y coordinates), and to choose the historically most stable identification criteria."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 11.6.1)

The question refers to a problem where data used for an object detection ML system was labelled incorrectly. This issue is most closely related to "accuracy issues." Here's a detailed explanation:

Accuracy Issues: The primary goal of labeling data in machine learning is to ensure that the model can accurately learn and make predictions based on the given labels. Incorrectly labeled data directly impacts the model's accuracy, leading to poor performance because the model learns incorrect patterns.

Why Not Other Options:

Security Issues: This pertains to data breaches or unauthorized access, which is not relevant to the problem of incorrect data labeling.

Privacy Issues: This concerns the protection of personal data and is not related to the accuracy of data labeling.

Bias Issues: While bias in data can affect model performance, it specifically refers to systematic errors or prejudices in the data rather than outright incorrect labeling.

The syllabus highlights that testing for unnecessary human intervention is a key focus for autonomous AI-based systems:

"For autonomous AI-based systems, testers must ensure that the system does not prompt for unnecessary human intervention, as this contradicts the autonomy concept."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 8.2, page 59 of 99)

The syllabus defines data acquisition as consisting of three steps:

“Data acquisition: The activity of acquiring data relevant to the business problem to be solved by an ML model, typically involving the activities of identifying, gathering and labelling data.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 4.1, page 33 of 99)

ï‚· Labelling Approaches:Among the options provided, pre-labeled datasets require the least time and effort because the data has already been labeled, eliminating the need for further manual or automated labeling efforts.

 Reference:ISTQB_CT-AI_Syllabus_v1.0, Section 4.5 Data Labelling for Supervised Learning, which discusses various approaches to data labeling, including pre-labeled datasets, and their associated time and effort requirements​​.

The syllabus states:

“Supervised learning… divides problems into two categories: classification and regression. Regression is used when the problem requires the ML model to predict a numeric output, for example predicting the age of a person based on their habits.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 3.1.1, Page 26 of 99)

The syllabus definesexplainabilityas the ability to understand how the AI-based system comes up with a particular result:

"Explainability is considered to be the ease with which users can determine how the AI-based system comes up with a particular result."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 2.7)