Amazon Web Services MLA-C01 - AWS Certified Machine Learning Engineer - Associate

When predicting continuous variables, such as apartment prices, it ' s essential to evaluate the model ' s performance using appropriate regression metrics. The Mean Absolute Error (MAE) is a widely used metric for this purpose.

Understanding Mean Absolute Error (MAE):

MAE measures the average magnitude of errors in a set of predictions, without considering their direction. It calculates the average absolute difference between predicted values and actual values, providing a straightforward interpretation of prediction accuracy.

Advantages of MAE:

Interpretability: MAE is expressed in the same units as the target variable, making it easy to understand.

Robustness to Outliers: Unlike metrics that square the errors (e.g., Mean Squared Error), MAE does not disproportionately penalize larger errors, making it more robust to outliers.

Comparison with Other Metrics:

Accuracy, AUC, F1 Score: These metrics are designed for classification tasks, where the goal is to predict discrete labels. They are not suitable for regression problems involving continuous target variables.

Mean Squared Error (MSE): While MSE also measures prediction errors, it squares the differences, giving more weight to larger errors. This can be useful in certain contexts but may be sensitive to outliers.

Conclusion:

For evaluating the performance of a model predicting apartment prices—a continuous variable—MAE is an appropriate and effective metric. It provides a clear indication of the average prediction error in the same units as the target variable, facilitating straightforward interpretation and comparison.

For near real-time anomaly detection on streaming IoT data, AWS recommends Amazon Managed Service for Apache Flink. Flink is designed for stateful, low-latency stream processing and is well suited for time-series sensor analytics.

A key requirement is application resilience during deployments. Managed Flink supports system rollback, which automatically reverts to the last stable application version if a new deployment fails. This capability ensures uninterrupted processing even if application bugs are introduced, meeting the company’s reliability requirement.

Manual rollback (Option B) introduces operational risk and delays. Amazon Data Firehose (Options C and D) is a delivery service and does not support complex anomaly detection logic or application version rollback.

Therefore, using Managed Flink with system rollback enabled is the correct solution.

This is a regression problem, where the target variable is continuous and numeric. AWS documentation clearly states that classification metrics such as accuracy and AUC are not appropriate for regression models.

Root Mean Square Error (RMSE) measures the square root of the average squared differences between predicted and actual values. RMSE penalizes larger errors more heavily, making it especially useful when large prediction errors are costly or undesirable.

SageMaker Canvas automatically selects regression metrics such as RMSE and MAE when building regression models. RMSE is widely used for time-based and numeric prediction problems, especially when evaluating long historical datasets.

Inference latency measures system performance, not model accuracy.

Therefore, Option D is the correct and AWS-verified answer.

The problem described is a patient segmentation use case, which is a classic example of unsupervised learning. The objective is to group patients with similar characteristics without predefined labels. AWS documentation clearly states that Amazon SageMaker k-means is designed specifically for clustering and segmentation tasks.

The SageMaker k-means algorithm groups data points into clusters based on feature similarity and requires the user to define the number of clusters using the k hyperparameter. This directly satisfies the requirement to “determine the number of groups by using hyperparameters.” AWS recommends k-means for applications such as customer segmentation, risk grouping, and pattern discovery in healthcare data.

Option A (XGBoost) is a supervised learning algorithm used for classification and regression. The max_depth hyperparameter controls tree complexity, not the number of groups, making it unsuitable for this task.

Option C (DeepAR) is a time-series forecasting algorithm optimized for predicting future values, not clustering patients.

Option D (Random Cut Forest) is an anomaly detection algorithm. While useful for identifying outliers or unusual patient behavior, it does not perform clustering or group segmentation.

AWS SageMaker documentation explicitly identifies k-means as the correct choice when the goal is to partition data into a predefined number of clusters using a tunable hyperparameter.

Therefore, Option B is the correct and AWS-verified answer.

When Model Monitor identifies data quality issues, it might be due to a shift in the data distribution compared to the original baseline. By creating a new baseline using the most recent production data and updating Model Monitor to evaluate against this baseline, the ML engineer ensures that the monitoring is aligned with the current data patterns. This approach mitigates false positives and reflects the updated data characteristics without immediately retraining the model.

The large gap between training accuracy (99%) and validation accuracy (82%) is a textbook case of overfitting. The model has learned patterns that fit the training data extremely well but do not generalize to unseen data.

AWS ML best practices recommend regularization techniques to address overfitting. Dropout layers randomly deactivate neurons during training, preventing the network from relying too heavily on specific paths. L1 and L2 regularization penalize large weights, reducing model complexity and improving generalization. k-fold cross-validation provides a more robust evaluation by training and validating the model across multiple data splits.

Option A increases complexity, which would worsen overfitting. Option C mixes valid ideas (dimensionality reduction) with unrelated changes (loss function choice) and is less targeted. Option D focuses on data quality but does not directly address model variance.

Therefore, implementing dropout, regularization, and k-fold cross-validation is the correct solution.

SageMaker Clarify is designed to provide explainability for ML models. It can analyze feature importance and explain how input features influence the model ' s predictions. By using Clarify with the deployed SageMaker model, the ML engineer can generate insights and present them to stakeholders to explain the sentiment analysis predictions effectively.

Option A is correct because AWS Glue provides the lowest-overhead managed pattern for this scenario: use a crawler to keep the AWS Glue Data Catalog updated for the S3 dataset, then use AWS Glue Data Quality to evaluate rules against that cataloged data. AWS documentation states that crawlers can automatically discover and catalog new or updated S3 data sources, reducing manual metadata management overhead. AWS Glue Data Quality is also documented as a managed, serverless capability for measuring and monitoring data quality.

This option is especially strong because the dataset is already in Amazon S3 and partitioned by date, and the requirement is simply to automatically check quality before the data is sent to QuickSight. AWS Glue Data Quality supports the Data Catalog as an entry point, and AWS documentation notes that this approach is intended for datasets that are already cataloged and for ongoing governance-style quality evaluation. The docs also show that scheduling is supported for Data Catalog-based quality checks. That means the company can run the crawler monthly, then evaluate a ruleset against the cataloged table without needing custom ETL code.

The other options add more operational work. Option B requires a custom PySpark job and custom functions, which is more maintenance-heavy than using catalog-based quality rules directly. Option C depends on bespoke Lambda scripts. Option D does not use the right service for data quality validation. Because the question asks for the least operational overhead, the most AWS-aligned answer is A: monthly Glue crawler plus Glue Data Quality rules.

Amazon SageMaker AI automatically collects aggregated metadata from training jobs to improve service reliability, performance, and operational insights. This metadata can include information such as algorithm usage, instance types, resource utilization, and job configuration details. However, AWS documentation clearly states that customers can opt out of SageMaker metadata collection to meet regulatory or compliance requirements.

SageMaker provides a supported mechanism to disable metadata tracking at the training job level. By explicitly opting out of metadata tracking when submitting training jobs—either through the AWS Management Console, AWS CLI, or SDK—the service will stop collecting aggregated metadata for those jobs. This option is specifically designed for customers with strict compliance, data residency, or regulatory constraints.

Option B is incorrect because running training jobs in a private subnet within a custom VPC controls network isolation, not service-level telemetry or metadata collection. Metadata collection occurs at the SageMaker service layer and is independent of VPC configuration.

Option C is also incorrect because encrypting training data with a customer-managed AWS KMS key protects data at rest and in transit but does not prevent SageMaker from collecting operational metadata about training jobs.

Option D is incorrect because AWS Nitro instances provide enhanced security and performance isolation at the infrastructure level but have no impact on SageMaker’s metadata collection mechanisms.

Therefore, opting out of metadata tracking for training jobs is the only solution that directly addresses the compliance requirement and is explicitly supported by AWS documentation.

AWS recommends Amazon SageMaker Model Monitor as the native service for detecting data drift, model drift, and bias drift in deployed ML models. Model Monitor continuously compares incoming inference data against a baseline dataset captured during training.

When Model Monitor detects drift beyond configured thresholds, it can emit Amazon CloudWatch events. These events can trigger an AWS Lambda function, which is a common AWS-documented pattern for orchestrating automated workflows such as model retraining.

This Lambda function can then initiate a SageMaker Pipeline execution, starting a retraining job with updated data. This architecture aligns with AWS best practices for building automated, event-driven ML pipelines.

Option A is incorrect because AWS Glue is designed for data cataloging and ETL, not for ML-specific drift detection. Option B is unnecessary and overly complex for this use case. Option D is incorrect because Amazon QuickSight anomaly detection is intended for business intelligence analytics, not ML model monitoring.

AWS documentation explicitly positions SageMaker Model Monitor + Lambda automation as the recommended approach for continuous ML monitoring and retraining.

Therefore, Option C is the correct and AWS-verified answer.