Google Professional-Data-Engineer - Google Professional Data Engineer Exam

Google Professional-Data-Engineer Premium Access Download Demo

Page: 1 / 6
Total 400 questions

Flowlogistic wants to use Google BigQuery as their primary analysis system, but they still have Apache Hadoop and Spark workloads that they cannot move to BigQuery. Flowlogistic does not know how to store the data that is common to both workloads. What should they do?

Store the common data in BigQuery as partitioned tables.

Store the common data in BigQuery and expose authorized views.

Store the common data encoded as Avro in Google Cloud Storage.

Store he common data in the HDFS storage for a Google Cloud Dataproc cluster.

Question # 2

Flowlogisticâ€™s management has determined that the current Apache Kafka servers cannot handle the data volume for their real-time inventory tracking system. You need to build a new system on Google Cloud Platform (GCP) that will feed the proprietary tracking software. The system must be able to ingest data from a variety of global sources, process and query in real-time, and store the data reliably. Which combination of GCP products should you choose?

Cloud Pub/Sub, Cloud Dataflow, and Cloud Storage

Cloud Pub/Sub, Cloud Dataflow, and Local SSD

Cloud Pub/Sub, Cloud SQL, and Cloud Storage

Cloud Load Balancing, Cloud Dataflow, and Cloud Storage

Question # 3

You have a data analyst team member who needs to analyze data by using BigQuery. The data analyst wants to create a data pipeline that would load 200 CSV files with an average size of 15MB from a Cloud Storage bucket into BigQuery daily. The data needs to be ingested and transformed before being accessed in BigQuery for analysis. You need to recommend a fully managed, no-code solution for the data analyst. What should you do?

Create a Cloud Run function and schedule it to run daily using Cloud Scheduler to load the data into BigQuery.

Use the BigQuery Data Transfer Service to load files from Cloud Storage to BigQuery, create a BigQuery job which transforms the data using BigQuery SQL and schedule it to run daily.

Build a custom Apache Beam pipeline and run it on Dataflow to load the file from Cloud Storage to BigQuery and schedule it to run daily using Cloud Composer.

Create a pipeline by using BigQuery pipelines and schedule it to load the data into BigQuery daily.

Explanation:

The requirements are for a daily scheduled load, ingest, and transformation, and specifically a fully managed, no-code solution.

Ingest (Load): The BigQuery Data Transfer Service (DTS) is the fully managed, serverless, and no-code solution for batch loading files (including CSV from Cloud Storage) into BigQuery on a schedule. This is the "ingest" part.

Transform: After loading the raw data into a staging table using DTS, the transformation can be done using BigQuery SQL. This transformation query can then be automated using a Scheduled Query in BigQuery, which is also a fully managed and no-code feature that runs on a schedule.

Fully Managed & No-Code: Both DTS for Cloud Storage and Scheduled Queries are native BigQuery features that are fully managed and configured through the console without requiring code, directly meeting the constraints.

Correcting other options:

A (Cloud Run + Script): Cloud Run requires writing a custom Python script, which violates the no-code requirement.

C (Dataflow + Apache Beam + Cloud Composer): This is a powerful, highly scalable ETL solution, but it requires writing custom code (Apache Beam) and requires setting up and managing a workflow orchestrator (Cloud Composer/Airflow), which violates both the fully managed (Dataflow is serverless, but the code/pipeline itself is custom and needs maintenance) and no-code requirements.

D (BigQuery pipelines): "BigQuery pipelines" is not a distinct, official product name in the Google Cloud documentation that fulfills a no-code scheduled ETL. The closest product is the combination of DTS and Scheduled Queries, as described in option B.

[Reference: Google Cloud Documentation on BigQuery Data Transfer Service and Scheduled Queries:, "The BigQuery Data Transfer Service automates data movement into BigQuery on a scheduled, managed basis... The BigQuery Data Transfer Service supports loading data from Cloud Storage in one of the following formats: Comma-separated values (CSV)..." (Source: What is BigQuery Data Transfer Service? and Introduction to Cloud Storage transfers), "A scheduled query is a query that BigQuery automatically runs at regular intervals. When you configure a scheduled query, you specify the GoogleSQL SELECT statement to run, the destination table for the query results, and the frequency of the query." (Source: Scheduling queries), This combination delivers a fully managed, no-code ELT (Extract-Load-Transform) pipeline., , , ]

Question # 4

You need to create a data pipeline that copies time-series transaction data so that it can be queried from within BigQuery by your data science team for analysis. Every hour, thousands of transactions are updated with a new status. The size of the intitial dataset is 1.5 PB, and it will grow by 3 TB per day. The data is heavily structured, and your data science team will build machine learning models based on this data. You want to maximize performance and usability for your data science team. Which two strategies should you adopt? Choose 2 answers.

Denormalize the data as must as possible.

Preserve the structure of the data as much as possible.

Use BigQuery UPDATE to further reduce the size of the dataset.

Develop a data pipeline where status updates are appended to BigQuery instead of updated.

Copy a daily snapshot of transaction data to Cloud Storage and store it as an Avro file. Use BigQueryâ€™s support for external data sources to query.

Question # 5

You are building a report-only data warehouse where the data is streamed into BigQuery via the streaming API Following Google's best practices, you have both a staging and a production table for the data How should you design your data loading to ensure that there is only one master dataset without affecting performance on either the ingestion or reporting pieces?

Have a staging table that is an append-only model, and then update the production table every three hourswith the changes written to staging

Have a staging table that is an append-only model, and then update the production table every ninetyminutes with the changes written to staging

Have a staging table that moves the staged data over to the production table and deletes the contents of thestaging table every three hours

Have a staging table that moves the staged data over to the production table and deletes the contents of the staging table every thirty minutes

Question # 6

You are running your BigQuery project in the on-demand billing model and are executing a change data capture (CDC) process that ingests data. The CDC process loads 1 GB of data every 10 minutes into a temporary table, and then performs a merge into a 10 TB target table. This process is very scan intensive and you want to explore options to enable a predictable cost model. You need to create a BigQuery reservation based on utilization information gathered from BigQuery Monitoring and apply the reservation to the CDC process. What should you do?

Create a BigQuery reservation for the job.

Create a BigQuery reservation for the service account running the job.

Create a BigQuery reservation for the dataset.

Create a BigQuery reservation for the project.

Explanation:

https://cloud.google.com/blog/products/data-analytics/manage-bigquery-costs-with-custom-quotas.

Here's why creating a BigQuery reservation for the project is the most suitable solution:

Project-Level Reservation:BigQuery reservations are applied at the project level. This means that the reserved slots (processing capacity) are shared across all jobs and queries running within that project. Since your CDC process is a significant contributor to your BigQuery usage, reserving slots for the entire project ensures that your CDC process always has access to the necessary resources, regardless of other activities in the project.

Predictable Cost Model:Reservations provide a fixed, predictable cost model. Instead of paying the on-demand price for each query, you pay a fixed monthly fee for the reserved slots. This eliminates the variability of costs associated with on-demand billing, making it easier to budget and forecast your BigQuery expenses.

BigQuery Monitoring:You can use BigQuery Monitoring to analyze the historical usage patterns of your CDC process and other queries within your project. This informationhelps you determine the appropriate amount of slots to reserve, ensuring that you have enough capacity to handle your workload while optimizing costs.

Why other options are not suitable:

A. Create a BigQuery reservation for the job:BigQuery does not support reservations at the individual job level. Reservations are applied at the project or assignment level.

B. Create a BigQuery reservation for the service account running the job:While you can create reservations for assignments (groups of users or service accounts), it's less efficient than a project-level reservation in this scenario. A project-level reservation covers all jobs within the project, regardless of the service account used.

C. Create a BigQuery reservation for the dataset:BigQuery does not support reservations at the dataset level.

By creating a BigQuery reservation for your project based on your utilization analysis, you can achieve a predictable cost model while ensuring that your CDC process and other queries have the necessary resources to run smoothly.

Question # 7

You are designing a Dataflow pipeline for a batch processing job. You want to mitigate multiple zonal failures at job submission time. What should you do?

Specify a worker region by using the â€”region flag.

Set the pipeline staging location as a regional Cloud Storage bucket.

Submit duplicate pipelines in two different zones by using the â€”zone flag.

Create an Eventarc trigger to resubmit the job in case of zonal failure when submitting the job.

Question # 8

You used Cloud Dataprep to create a recipe on a sample of data in a BigQuery table. You want to reuse this recipe on a daily upload of data with the same schema, after the load job with variable execution time completes. What should you do?

Create a cron schedule in Cloud Dataprep.

Create an App Engine cron job to schedule the execution of the Cloud Dataprep job.

Export the recipe as a Cloud Dataprep template, and create a job in Cloud Scheduler.

Export the Cloud Dataprep job as a Cloud Dataflow template, and incorporate it into a Cloud Composer job.

Question # 9

You are collecting loT sensor data from millions of devices across the world and storing the data in BigQuery. Your access pattern is based on recent data tittered by location_id and device_version with the following query:

You want to optimize your queries for cost and performance. How should you structure your data?

Partition table data by create_date, location_id and device_version

Partition table data by create_date cluster table data by tocation_id and device_version

Cluster table data by create_date location_id and device_version

Cluster table data by create_date, partition by location and device_version

Question # 10

MJTelcoâ€™s Google Cloud Dataflow pipeline is now ready to start receiving data from the 50,000 installations. You want to allow Cloud Dataflow to scale its compute power up as required. Which Cloud Dataflow pipeline configuration setting should you update?

The zone

The number of workers

The disk size per worker

The maximum number of workers

Spring Sale Special Limited Time 70% Discount Offer - Ends in 0d 00h 00m 00s - Coupon code: xmas50

Google Professional-Data-Engineer - Google Professional Data Engineer Exam

The Answer Is:

The Answer Is:

The Answer Is:

Explanation:

The Answer Is:

The Answer Is:

The Answer Is:

Explanation:

The Answer Is:

Explanation:

The Answer Is:

The Answer Is:

The Answer Is: