ISTQB CT-AI - ISTQB Certified Tester AI Testing Exam

The syllabus defines data acquisition as consisting of three steps:

“Data acquisition: The activity of acquiring data relevant to the business problem to be solved by an ML model, typically involving the activities of identifying, gathering and labelling data.”

(Reference: ISTQB CT-AI Syllabus v1.0, Section 4.1, page 33 of 99)

Section4.3 – Test Levels for AI Systemsclearly defines ML model testing as the level at which testers evaluate whether an ML model fulfills itsfunctional performance criteria, including accuracy, precision, recall, F1, robustness, stability, and fairness. Therefore, Option C is the correct and syllabus-aligned statement.

Option A is incorrect because input data testing focuses onvalidity and correctness of data entering the model, not interactions with all system components. Option B is incorrect: acceptance testing in the syllabus focuses primarily onbusiness and stakeholder requirements, not specifically explainability. Explainability testing may occur at multiple levels depending on context. Option D is also incorrect because API testing belongs tointegration testing, not system testing, even when AI is consumed as a service.

Thus,Option Cis the only statement that precisely matches syllabus definitions.

The ISTQB CT-AI syllabus explains ML performance metrics in Section3.2 – Evaluating ML Models. Forclustering, which is an unsupervised learning method, the syllabus lists metrics such asintra-cluster distance,inter-cluster distance, and coherence measures. Intra-cluster metrics evaluate how close data points are within a cluster, which directly corresponds to Option A.

Option B is incorrect becauseR-squaredis a regression metric measuring goodness-of-fit, not classification performance, and has no connection to ROC curves. Option C is wrong because thesilhouette coefficientis also a clustering metric, measuring cohesion vs. separation—not regression accuracy. Option D is incorrect because ROC curves evaluatebinary or multiclass classification, not clustering.

Thus, OptionAis the only accurate statement based on the syllabus.

The syllabus defines bias as:

"Bias is the systematic difference in treatment of certain objects, people or groups in comparison to others."

It also discusses:

"Sample bias can occur if the data used for training the model does not represent the operational environment, or if some relevant faulty conditions are excluded deliberately."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 7.6 and 8.3)

To calculate the accuracy, recall, and specificity from the confusion matrix provided, we use the following formulas:

Confusion Matrix:

Actually Rotten: 45 (True Positive), 8 (False Positive)

Actually Fresh: 5 (False Negative), 42 (True Negative)

Accuracy:

Accuracy is the proportion of true results (both true positives and true negatives) in the total population.

Formula: Accuracy=TP+TNTP+TN+FP+FN\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}Accuracy=TP+TN+FP+FNTP+TN​

Calculation: Accuracy=45+4245+42+8+5=87100=0.87\text{Accuracy} = \frac{45 + 42}{45 + 42 + 8 + 5} = \frac{87}{100} = 0.87Accuracy=45+42+8+545+42​=10087​=0.87

Recall (Sensitivity):

Recall is the proportion of true positive results in the total actual positives.

Formula: Recall=TPTP+FN\text{Recall} = \frac{TP}{TP + FN}Recall=TP+FNTP​

Calculation: Recall=4545+5=4550=0.9\text{Recall} = \frac{45}{45 + 5} = \frac{45}{50} = 0.9Recall=45+545​=5045​=0.9

Specificity:

Specificity is the proportion of true negative results in the total actual negatives.

Formula: Specificity=TNTN+FP\text{Specificity} = \frac{TN}{TN + FP}Specificity=TN+FPTN​

Calculation: Specificity=4242+8=4250=0.84\text{Specificity} = \frac{42}{42 + 8} = \frac{42}{50} = 0.84Specificity=42+842​=5042​=0.84

Therefore, the correct combinations of accuracy, recall, and specificity are 0.87, 0.9, and 0.84 respectively.

The syllabus explains that EDA can be used to analyze data to identify outliers and unusual patterns, which can indicate adversarial attacks like data poisoning:

"Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers."

(Reference: ISTQB CT-AI Syllabus v1.0, Section 9.1.2, page 67 of 99)