Amazon Web Services AIF-C01 - AWS Certified AI Practitioner Exam

The ecommerce company’s chatbot, powered by AI, automates customer order submissions and is accessible 24/7 via the website. Prompt injection is an AI system input vulnerability where malicious users craft inputs to manipulate the chatbot’s behavior, such as bypassing safeguards or accessing unauthorized information. This vulnerability must be resolved before the chatbot is made available to ensure security.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

" Prompt injection is a vulnerability in AI systems, particularly chatbots, where malicious inputs can manipulate the model’s behavior, potentially leading to unauthorized actions or harmful outputs. Implementing guardrails and input validation can mitigate this risk. "

(Source: AWS Bedrock User Guide, Security Best Practices)

Detailed Explanation:

Option A: Data leakageData leakage refers to the unintended exposure of sensitive data during model training or inference, not an input vulnerability affecting a chatbot’s operation.

Option B: Prompt injectionThis is the correct answer. Prompt injection is a critical input vulnerability for chatbots, where malicious prompts can exploit the AI to produce harmful or unauthorized responses, a risk that must be addressed before launch.

Option C: Large language model (LLM) hallucinationsLLM hallucinations refer to the model generating incorrect or ungrounded responses, which is an output issue, not an input vulnerability.

Option D: Concept driftConcept drift occurs when the data distribution changes over time, affecting model performance. It is not an input vulnerability but a long-term performance issue.

Amazon Bedrock offers an on-demand pricing model that provides flexibility without long-term commitments. This model allows companies to pay only for the resources they use, which is ideal for a limited budget and offers flexibility.

Option A (Correct): " On-Demand " : This is the correct answer because on-demand pricing allows the company to use Amazon Bedrock without any long-term commitments and to manage costs according to their budget.

Option B: " Model customization " is a feature, not a pricing model.

Option C: " Provisioned Throughput " involves reserving capacity ahead of time, which might not offer the desired flexibility and could lead to higher costs if the capacity is not fully used.

Option D: " Spot Instance " is a pricing model for EC2 instances and does not apply to Amazon Bedrock.

AWS AI Practitioner References:

AWS Pricing Models for Flexibility: On-demand pricing is a key AWS model for services that require flexibility and no long-term commitment, ensuring cost-effectiveness for projects with variable usage patterns.

Comprehensive and Detailed Explanation From Exact AWS AI documents:

The temperature parameter controls randomness in model outputs.

AWS generative AI guidance explains:

Higher temperature → more randomness and creativity

Lower temperature → more deterministic and predictable outputs

To increase diversity and creativity, the temperature should be increased.

Why the other options are incorrect:

Lower Top K (B) reduces output diversity.

Response length (C) affects size, not creativity.

Prompt length (D) does not directly control randomness.

AWS AI document references:

Inference Parameters for Foundation Models

Controlling Creativity in LLMs

Text Generation Configuration on AWS

The airline company aims to build a conversational AI assistant using large language models (LLMs) and a knowledge base to create a text-based chatbot with minimal development effort. Retrieval Augmented Generation (RAG) on Amazon Bedrock is an ideal solution because it combines LLMs with a knowledge base to provide accurate, contextually relevant responses without requiring extensive model training or custom development. RAG retrieves relevant information from a knowledge base and uses an LLM to generate responses, simplifying the development process.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

" Retrieval Augmented Generation (RAG) in Amazon Bedrock enables developers to build conversational AI applications by combining foundation models with external knowledge bases. This approach minimizes development effort by leveraging pre-trained models and integrating them with data sources, such as FAQs or databases, to provide accurate and contextually relevant responses. "

(Source: AWS Bedrock User Guide, Retrieval Augmented Generation)

Detailed Explanation:

Option A: Train models on Amazon SageMaker Autopilot.SageMaker Autopilot is designed for automated machine learning (AutoML) tasks like classification or regression, not for building conversational AI with LLMs and knowledge bases. It requires significant data preparation and is not optimized for chatbot development, making it less suitable.

Option B: Develop a Retrieval Augmented Generation (RAG) agent by using Amazon Bedrock.This is the correct answer. RAG on Amazon Bedrock allows the company to use pre-trained LLMs and integrate them with a knowledge base (e.g., flight schedules or FAQs) to build a chatbot with minimal effort. It avoids the need for extensive training or coding, aligning with the requirement for least development effort.

Option C: Create a Python application by using Amazon Q Developer.While Amazon Q Developer can assist with code generation, building a chatbot from scratch in Python requires significant development effort, including integrating LLMs and a knowledge base manually, which is more complex than using RAG on Bedrock.

Option D: Fine-tune models on Amazon SageMaker Jumpstart.Fine-tuning models on SageMaker Jumpstart requires preparing training data and customizing LLMs, which involves more effort than using a pre-built RAG solution on Bedrock. This option is not the least effort-intensive.

Tokens in generative AI models are the smallest units that the model processes, typically representing words, subwords, or characters. They are essential for the model to understand and generate language, breaking down text into manageable parts for processing.

Option A (Correct): " Tokens are the basic units of input and output that a generative AI model operates on, representing words, subwords, or other linguistic units " : This is the correct definition of tokens in the context of generative AI models.

Option B: " Mathematical representations of words " describes embeddings, not tokens.

Option C: " Pre-trained weights of a model " refers to the parameters of a model, not tokens.

Option D: " Prompts or instructions given to a model " refers to the queries or commands provided to a model, not tokens.

AWS AI Practitioner References:

Understanding Tokens in NLP: AWS provides detailed explanations of how tokens are used in natural language processing tasks by AI models, such as in Amazon Comprehend and other AWS AI services.

Comprehensive and Detailed Explanation From Exact AWS AI documents:

Amazon Macie uses machine learning to:

Discover sensitive data such as PII

Classify data stored in Amazon S3

Help prevent unintended data exposure

AWS security guidance recommends Macie before data sharing to ensure compliance and privacy protection.

Why the other options are incorrect:

Kendra (A) is a search service.

Textract (C) extracts text from documents.

Data Exchange (D) shares datasets, not analyzes sensitivity.

AWS AI document references:

Amazon Macie Overview

Protecting Sensitive Data in S3

Data Privacy and Governance on AWS

Domain adaptation fine-tuning involves training a foundation model (FM) further using a specific dataset that includes domain-specific terminology and content, such as scientific terms in research papers. This process allows the model to better understand and handle complex terminology, improving its performance on specialized tasks.

Option B (Correct): " Use domain adaptation fine-tuning to adapt the FM to complex scientific terms " : This is the correct answer because fine-tuning the model on domain-specific data helps it learn and adapt to the specific language and terms used in the research papers, resulting in better performance.

Option A: " Use few-shot prompting to define how the FM can answer the questions " is incorrect because while few-shot prompting can help in certain scenarios, it is less effective than fine-tuning for handling complex domain-specific terms.

Option C: " Change the FM inference parameters " is incorrect because adjusting inference parameters will not resolve the issue of the model ' s lack of understanding of complex scientific terminology.

Option D: " Clean the research paper data to remove complex scientific terms " is incorrect because removing the complex terms would result in the loss of important information and context, which is not a viable solution.

AWS AI Practitioner References:

Domain Adaptation in Amazon Bedrock: AWS recommends fine-tuning models with domain-specific data to improve their performance on specialized tasks involving unique terminology.

The business goal identification phase of the ML lifecycle involves defining the objectives of the project and understanding the requirements, including compliance and regulatory considerations. This phase ensures the ML solution aligns with legal and organizational standards before proceeding to technical stages like data collection or model training.

Exact Extract from AWS AI Documents:

From the AWS AI Practitioner Learning Path:

" The business goal identification phase involves defining the problem to be solved, identifying success metrics, and determining compliance and regulatory requirements to ensure the ML solution adheres to legal and organizational standards. "

(Source: AWS AI Practitioner Learning Path, Module on Machine Learning Lifecycle)

Detailed Explanation:

Option A: Feature engineeringFeature engineering involves creating or selecting features for model training, which occurs after compliance requirements are identified. It does not address regulatory concerns.

Option B: Model trainingModel training focuses on building the ML model using data, not on determining compliance or regulatory requirements.

Option C: Data collectionData collection involves gathering data for training, but compliance and regulatory requirements (e.g., data privacy laws) are defined earlier in the business goal identification phase.

Option D: Business goal identificationThis is the correct answer. This phase ensures that compliance and regulatory requirements are considered at the outset, shaping the entire ML project.

The company is developing an ML model to predict customer churn, a binary classification task (churn or no churn). The F1 score is an evaluation metric that balances precision and recall, making it suitable for assessing the performance of binary classification models, especially when dealing with imbalanced datasets, which is common in churn prediction.

Exact Extract from AWS AI Documents:

From the Amazon SageMaker Developer Guide:

" The F1 score is a metric for evaluating binary classification models, combining precision and recall into a single value. It is particularly useful for tasks like churn prediction, where class imbalance may exist, ensuring the model performs well on both positive and negative classes. "

(Source: Amazon SageMaker Developer Guide, Model Evaluation Metrics)

Detailed Explanation:

Option A: F1 scoreThis is the correct answer. The F1 score is ideal for binary classification tasks like churn prediction, as it measures the model’s ability to correctly identify both churners and non-churners.

Option B: Mean squared error (MSE)MSE is used for regression tasks to measure the average squared difference between predicted and actual values, not for binary classification.

Option C: R-squaredR-squared is a metric for regression models, indicating how well the model explains the variability of the target variable. It is not applicable to classification tasks.

Option D: Time used to train the modelTraining time is not an evaluation metric for model performance; it measures the duration of training, not the model’s accuracy or effectiveness.