Amazon Web Services AIF-C01 - AWS Certified AI Practitioner Exam

The ML model shows 90% recall on training data but only 40% recall on unseen testing data, indicating a significant performance drop. This discrepancy suggests the model has learned the training data too well, including noise and specific patterns that do not generalize to new data, which is a classic sign of overfitting.

Exact Extract from AWS AI Documents:

From the Amazon SageMaker Developer Guide:

" Overfitting occurs when a model performs well on training data but poorly on unseen test data, as it has learned patterns specific to the training set, including noise, that do not generalize. A large gap between training and testing performance metrics, such as recall, is a common indicator of overfitting. "

(Source: Amazon SageMaker Developer Guide, Model Evaluation and Overfitting)

Detailed Explanation:

Option A: The model is overfitting on the training data.This is the correct answer. The significant drop in recall from 90% (training) to 40% (testing) indicates the model is overfitting, as it performs well on training data but fails to generalize to unseen data.

Option B: The model is underfitting on the training data.Underfitting occurs when the model performs poorly on both training and testing data due to insufficient learning. With 90% recall on training data, the model is not underfitting.

Option C: The model has insufficient training data.Insufficient training data could lead to poor performance, but the high recall on trainingdata (90%) suggests the model has learned the training data well, pointing to overfitting rather than a lack of data.

Option D: The model has insufficient testing data.Insufficient testing data might lead to unreliable test metrics, but it does not explain the large performance gap between training and testing, which is more indicative of overfitting.

The correct answer is B because explainable AI (XAI) provides transparency into how models reach specific decisions. According to AWS documentation, techniques such as SHAP values (SHapley Additive exPlanations) or LIME can identify which input features (e.g., income, debt ratio, or credit history) most influenced a model’s prediction. This helps financial institutions comply with fairness and transparency requirements under regulatory frameworks like the Equal Credit Opportunity Act. AWS SageMaker Clarify is a built-in service that offers explainability reports and bias detection to enhance trust. Rule-based systems and UIs alone do not satisfy transparency standards, and accuracy improvements do not replace explainability. By implementing explainable AI, customers can understand and trust credit limit decisions, reducing bias concerns and ensuring compliance.

Referenced AWS AI/ML Documents and Study Guides:

Amazon SageMaker Clarify Documentation – Explainability and Feature Attribution

AWS Responsible AI Practices – Transparency and Accountability

Amazon SageMaker Clarify provides built-in tools to detect bias in data and models, and to generate detailed explainability reports for model predictions, including SHAP values and feature importance.

A is correct:

“Amazon SageMaker Clarify provides bias detection, explainability for ML models, and comprehensive reports to satisfy regulatory and ethical requirements.”

(Reference: Amazon SageMaker Clarify Overview)

B (Data Wrangler) is for data preparation, not bias/explainability.

C (Model Cards) document models, but don’t detect bias or explain predictions.

D (AI Service Cards) provide transparency for AWS AI services, not custom model explainability.

ï‚· Block input prompts or model responses that contain harmful content such as hate, insults, violence, or misconduct:Content filters

ï‚· Avoid subjects related to illegal investment advice or legal advice:Denied topics

ï‚· Detect and block specific offensive terms:Word filters

 Detect and filter out information in the model’s responses that is not grounded in the provided source information:Contextual grounding check

The company is using a generative AI model on Amazon Bedrock and needs to mitigate undesirable and potentially harmful content in the model’s responses. Amazon Bedrock provides several guardrail mechanisms, including content filters, denied topics, word filters, and contextual grounding checks, to ensure safe and accurate outputs. Each mitigation action in the hotspot aligns with a specific Bedrock filter policy, and each policy must be used exactly once.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

* " Amazon Bedrock guardrails provide mechanisms to control model outputs, including:

Content filters: Block harmful content such as hate speech, violence, or misconduct.

Denied topics: Prevent the model from generating responses on specific subjects, such as illegal activities or advice.

Word filters: Detect and block specific offensive or inappropriate terms.

Contextual grounding check: Ensure responses are grounded in the provided source information, filtering out ungrounded or hallucinated content. " *(Source: AWS Bedrock User Guide, Guardrails for Responsible AI)

Detailed Explanation:

Block input prompts or model responses that contain harmful content such as hate, insults, violence, or misconduct: Content filtersContent filters in Amazon Bedrock are designed to detect and block harmful content, such as hate speech, insults, violence, or misconduct, ensuring the model’s outputs are safe and appropriate. This matches the first mitigation action.

Avoid subjects related to illegal investment advice or legal advice: Denied topicsDenied topics allow users to specify subjects the model should avoid, such as illegal investment advice or legal advice, which could have regulatory implications. This policy aligns with the second mitigation action.

Detect and block specific offensive terms: Word filtersWord filters enable the detection and blocking of specific offensive or inappropriate terms defined by the user, making them ideal for this mitigation action focused on specific terms.

Detect and filter out information in the model’s responses that is not grounded in the provided source information: Contextual grounding checkThe contextual grounding check ensures that the model’s responses are based on the provided source information, filtering out ungrounded or hallucinated content. This matches the fourth mitigation action.

Hotspot Selection Analysis:

The hotspot lists four mitigation actions, each with the same dropdown options: " Select..., " " Content filters, " " Contextual grounding check, " " Denied topics, " and " Word filters. " The correct selections are:

First action: Content filters

Second action: Denied topics

Third action: Word filters

Fourth action: Contextual grounding check

Each filter policy is used exactly once, as required, and aligns with Amazon Bedrock’s guardrail capabilities.

Exploratory data analysis (EDA) involves understanding the data by visualizing it, calculating statistics, and creating correlation matrices. This stage helps identify patterns, relationships, and anomalies in the data, which can guide further steps in the ML pipeline.

Option C (Correct): " Exploratory data analysis " : This is the correct answer as the tasks described (correlation matrix, calculating statistics, visualizing data) are all part of the EDA process.

Option A: " Data pre-processing " is incorrect because it involves cleaning and transforming data, not initial analysis.

Option B: " Feature engineering " is incorrect because it involves creating new features from raw data, not analyzing the data ' s existing structure.

Option D: " Hyperparameter tuning " is incorrect because it refers to optimizing model parameters, not analyzing the data.

AWS AI Practitioner References:

Stages of the Machine Learning Pipeline: AWS outlines EDA as the initial phase of understanding and exploring data before moving to more specific preprocessing, feature engineering, and model training stages.

Adding a role description to the prompt context is a straightforward way to instruct the generative AI model to adjust its response style based on the user ' s age range. This method requires minimal implementation effort as it does not involve additional training or complex logic.

Option B (Correct): " Add a role description to the prompt context that instructs the model of the age range that the response should target " : This is the correct answer because it involves the least implementation effort while effectively guiding the model to tailor responses according to the age range.

Option A: " Fine-tune the model by using additional training data " is incorrect because it requires significant effort in gathering data and retraining the model.

Option C: " Use chain-of-thought reasoning " is incorrect as it involves complex reasoning that may not directly address the need to adjust response style based on age.

Option D: " Summarize the response text depending on the age of the user " is incorrect because it involves additional processing steps after generating the initial response, increasing complexity.

AWS AI Practitioner References:

Prompt Engineering Techniques on AWS: AWS recommends using prompt context effectively to guide generative models in providing tailored responses based on specific user attributes.

The airline company aims to build a conversational AI assistant using large language models (LLMs) and a knowledge base to create a text-based chatbot with minimal development effort. Retrieval Augmented Generation (RAG) on Amazon Bedrock is an ideal solution because it combines LLMs with a knowledge base to provide accurate, contextually relevant responses without requiring extensive model training or custom development. RAG retrieves relevant information from a knowledge base and uses an LLM to generate responses, simplifying the development process.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

" Retrieval Augmented Generation (RAG) in Amazon Bedrock enables developers to build conversational AI applications by combining foundation models with external knowledge bases. This approach minimizes development effort by leveraging pre-trained models and integrating them with data sources, such as FAQs or databases, to provide accurate and contextually relevant responses. "

(Source: AWS Bedrock User Guide, Retrieval Augmented Generation)

Detailed Explanation:

Option A: Train models on Amazon SageMaker Autopilot.SageMaker Autopilot is designed for automated machine learning (AutoML) tasks like classification or regression, not for building conversational AI with LLMs and knowledge bases. It requires significant data preparation and is not optimized for chatbot development, making it less suitable.

Option B: Develop a Retrieval Augmented Generation (RAG) agent by using Amazon Bedrock.This is the correct answer. RAG on Amazon Bedrock allows the company to use pre- trained LLMs and integrate them with a knowledge base (e.g., flight schedules or FAQs) to build a chatbot with minimal effort. It avoids the need for extensive training or coding, aligning with the requirement for least development effort.

Option C: Create a Python application by using Amazon Q Developer.While Amazon Q Developer can assist with code generation, building a chatbot from scratch in Python requires significant development effort, including integrating LLMs and a knowledge base manually, which is more complex than using RAG on Bedrock.

Option D: Fine-tune models on Amazon SageMaker Jumpstart.Fine-tuning models on SageMaker Jumpstart requires preparing training data and customizing LLMs, which involves more effort than using a pre-built RAG solution on Bedrock. This option is not the least effort-intensive.

Class imbalance in a training dataset can cause ML models to favor overrepresented groups, leading to biased predictions. The AWS AI Practitioner guide and SageMaker Clarify documentation emphasize the need to identify and mitigate class imbalance to ensure fairness and unbiased model outcomes.

D is correct: By measuring class imbalance and adapting the training process (e.g., through oversampling, undersampling, or using class weights), organizations can improve fairness and reduce bias across demographic groups.

A (reducing data size) could worsen bias by removing potentially useful diverse data.

B (consistency with historical results) might reinforce existing biases.

C (separate models) is not scalable and can introduce other fairness issues.

“To reduce bias, examine class imbalance in your training data and use techniques to ensure all groups are fairly represented.”

(Reference: AWS SageMaker Clarify: Mitigating Bias, AWS Responsible AI)

Explainable AI (XAI) techniques such as SHAP (SHapley values) or feature attribution provide transparency by showing which input factors influenced decisions.

A is not scalable for complex use cases.

C does not guarantee real interpretability.

D ignores the regulatory need for explainability.

📖 Reference:

AWS SageMaker Clarify – Explainable AI