Amazon Web Services AIF-C01 - AWS Certified AI Practitioner Exam

The verified answer is C. Use the LLM-as-a-judge approach with a rubric across prompts to compare model outputs. The question asks for a scalable way to compare two foundation models for chat summarization based on correctness and completeness. AWS documentation for Amazon Bedrock model evaluation states that an evaluation job using a judge model uses an LLM to score another model’s responses and provide an explanation of how it scored each prompt and response pair. This is specifically designed to compare model outputs across prompt datasets at scale, without requiring every output to be manually scored by humans.

AWS Bedrock documentation also lists built-in metrics for model evaluation jobs that use an LLM as a judge. The valid built-in metrics include Correctness and Completeness, which directly match the evaluation criteria in the question. Therefore, using LLM-as-a-judge with a rubric across prompts is the best fit because it evaluates the actual summaries produced by each model against the desired quality dimensions.

Option A is partially valid as an evaluation method, but it is not the most scalable approach. Human reviewers can provide high-quality judgments, but manual review becomes expensive and slow when comparing many prompts, outputs, and models. Option B is incorrect because latency and throughput measure operational performance, not whether a summary is correct or complete. A fast model can still produce incomplete or inaccurate summaries. Option D is incorrect because external benchmark rankings are generic. They do not prove that a model performs best on the company’s specific chat summarization workload.

The decisive terms are “scalable,” “correctness,” and “completeness.” Amazon Bedrock LLM-as-a-judge evaluation directly supports those dimensions, so option C is correct.

The company wants to enhance the response quality of an LLM for complex problem-solving tasks requiring detailed reasoning and step-by-step explanations. Chain-of-thought prompting encourages the LLM to break down the problem into intermediate steps, providing a clear reasoning process before arriving at the final answer, which is ideal for this requirement.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

" Chain-of-thought prompting improves the reasoning capabilities of large language models by encouraging them to break down complex tasks into intermediate steps, providing a step-by-step explanation that leads to the final answer. This technique is particularly effective for problem-solving tasks requiring detailed reasoning. "

(Source: AWS Bedrock User Guide, Prompt Engineering Techniques)

Detailed Explanation:

Option A: Few-shot promptingFew-shot prompting provides a few examples to guide the LLM but does not explicitly encourage step-by-step reasoning or detailed explanations.

Option B: Zero-shot promptingZero-shot prompting relies on the LLM’s pre-trained knowledge without examples, making it less effective for complex tasks requiring detailed reasoning.

Option C: Directional stimulus promptingDirectional stimulus prompting is not a standard technique in AWS documentation, likely a distractor, and does not address step-by-step reasoning.

Option D: Chain-of-thought promptingThis is the correct answer. Chain-of-thought prompting enhances response quality for complex tasks by guiding the LLM to reason step-by-step, providing detailed explanations.

Amazon SageMaker endpoints provide fully managed, serverless model deployment for real-time and batch predictions, allowing companies to deploy ML models without handling any servers or infrastructure management.

D is correct: SageMaker endpoints let you deploy, scale, and monitor ML models with no infrastructure overhead.

A and B require infrastructure management.

C (CloudFront/S3) is not for model deployment, but for static content delivery.

“Amazon SageMaker endpoints allow you to deploy machine learning models for inference without the need to manage underlying infrastructure.”

(Reference: AWS SageMaker Model Deployment, AWS Certified AI Practitioner Study Guide)

Amazon SageMaker Ground Truth is designed to support human evaluation and labeling workflows using both internal teams and external workforces. AWS documentation states that Ground Truth enables organizations to build high-quality labeled datasets and conduct human reviews using private workforces, vendor-managed workforces, or third-party providers.

In this scenario, the company needs to evaluate toxicity in foundation model outputs, which requires nuanced human judgment. AWS highlights that Ground Truth supports tasks such as content moderation, sentiment analysis, and safety evaluation, making it well suited for assessing harmful or toxic content generated by models.

Ground Truth provides managed tooling for task distribution, reviewer instructions, quality control, and auditability. It allows companies to seamlessly scale operations by combining internal reviewers for sensitive data with external reviewers for higher-volume workloads, while maintaining consistent review standards.

The other options do not meet the requirement. Amazon Bedrock Agents orchestrate interactions with foundation models but do not manage human review workflows. Amazon Comprehend Custom focuses on training NLP models, not evaluating FM outputs. Amazon SageMaker JumpStart provides pretrained models and solutions, not human evaluation pipelines.

AWS positions SageMaker Ground Truth as a core service for human-in-the-loop machine learning and responsible AI, making it the correct choice for toxicity evaluation using both internal and external reviewers.

Dataset 1: A dataset that contains text-based customer reviews → Natural language processing (NLP)

NLP is designed for analyzing text (sentiment analysis, text classification, etc.).

Dataset 2: A dataset that contains images of animals labeled with their species names → Computer vision

Computer vision models classify or detect objects in images.

Dataset 3: A dataset that contains daily sales volumes for products → Time series forecasting

Time series forecasting predicts future values based on historical sequential data (like sales, demand, stock prices).

Regression predicts continuous numerical values (e.g., stock prices, temperatures).

A is classification (genre selection).

B is clustering.

D is generative AI/computer vision.

📖 Reference:

AWS ML Glossary – Regression

AWS HealthScribe provides speech-to-text and medical documentation generation, specifically designed for healthcare applications.

Amazon Polly is text-to-speech, not speech-to-text.

Amazon Rekognition is computer vision.

Amazon Q Developer is a generative AI assistant for developers, not healthcare.

📖 Reference:

AWS Documentation – AWS HealthScribe

Amazon Q Developer is a tool designed to assist developers in increasing productivity by generating code snippets, managing reference tracking, and handling open-source license tracking. These features help developers by automating parts of the software development process.

Option A (Correct): " Create software snippets, reference tracking, and open-source license tracking " : This is the correct answer because these are key features that help developers streamline and automate tasks, thus improving productivity.

Option B: " Run an application without provisioning or managing servers " is incorrect as it refers to AWS Lambda or AWS Fargate, not Amazon Q Developer.

Option C: " Enable voice commands for coding and providing natural language search " is incorrect because this is not a function of Amazon Q Developer.

Option D: " Convert audio files to text documents by using ML models " is incorrect as this refers to Amazon Transcribe, not Amazon Q Developer.

AWS AI Practitioner References:

Amazon Q Developer Features: AWS documentation outlines how Amazon Q Developer supports developers by offering features that reduce manual effort and improve efficiency.

The correct answer is B – Use Amazon Bedrock batch inference, which allows asynchronous generation of large-scale model outputs through APIs without requiring low-latency performance. According to AWS Bedrock documentation, batch inference is ideal for high-volume workloads that can tolerate delay, such as bulk content generation or summarization jobs. Unlike real-time inference, it processes requests in bulk, reducing cost and operational load. AWS handles the queuing, processing, and scaling automatically. Bedrock Agents (option C) are for workflow orchestration, not large-scale generation. AWS Lambda (option D) can automate tasks but is not optimized for high-volume LLM calls. Batch inference provides cost efficiency, scalability, and simplicity for delayed, asynchronous generation needs.

Referenced AWS AI/ML Documents and Study Guides:

Amazon Bedrock Developer Guide – Batch Inference

AWS ML Specialty Study Guide – Scalable Inference Options

Fine-tuning is the process of training a pre-trained LLM with a labeled dataset specific to a desired task—in this case, mapping user messages to JSON outputs. Fine-tuning leverages supervised learning to specialize the model’s outputs.

C is correct:

“Fine-tuning is a supervised learning approach in which a model is further trained on a custom, labeled dataset to adapt to a specific use case.”

(Reference: Amazon Bedrock Fine-Tuning, AWS Certified AI Practitioner Study Guide)

A is incorrect—unsupervised learning does not use labeled data.

B (continued pre-training) uses unlabeled data.

D (RLHF) uses reward signals and human feedback, not direct labeled input/output pairs.