[Qwiklabs]Perform Foundational Data, ML, and AI Tasks in Google Cloud: Challenge Lab

Task 1: Run a simple Dataflow job

1. Go to the cloud storage bucket and create a bucket with the name as mentioned in the lab instruction page.
2. Go to the BigQuery and create the Dataset and give dataset name as given in the lab instruction page
3. Click on Dataset you created from left side menu
4. Then click on + Create table from right side menu
5. Right side please click on create table option:

Create table from: Google cloud storage Create file from GCS bucket: cloud-training/gsp323/lab.csv Table Name: as given in the lab instructions File format: CSV Under Schema: Enable Edit as text and paste below code and click on create table

[
{“type”:”STRING”,”name”:”guid”},
{“type”:”BOOLEAN”,”name”:”isActive”},
{“type”:”STRING”,”name”:”firstname”},
{“type”:”STRING”,”name”:”surname”},
{“type”:”STRING”,”name”:”company”},
{“type”:”STRING”,”name”:”email”},
{“type”:”STRING”,”name”:”phone”},
{“type”:”STRING”,”name”:”address”},
{“type”:”STRING”,”name”:”about”},
{“type”:”TIMESTAMP”,”name”:”registered”},
{“type”:”FLOAT”,”name”:”latitude”},
{“type”:”FLOAT”,”name”:”longitude”}
]

Click on create table.

1. Goto the dataflow from navigation menu & click on create job template
2. Job name: job123 or any of your choice
3. Region: us-central1
4. Dataflow batch template: Text Files on Cloud Storage to BigQuery under “Process Data in Bulk (batch)
5. Add the other details for for the template from the lab instruction page

Click on Run Job.

Task 2. Run a simple Dataproc job

1. From the Navigation menu select Dataproc

Click on Create Cluster option

Select Create cluster on VM

Region# Region Name

Click on Create

2. Click on Cluster name that we have created

Select VM instances

Click on SSH option available

In the New window of SSH, click on Connect

Paste command

 hdfs dfs -cp gs://cloud-training/gsp323/data.txt /data.txt

3. Under Dataproc panel at left side select Job

Select Submit Job

Select cluster name from dropdown list of Cluster

Job type# Spark

Main class or jar# org.apache.spark.examples.SparkPageRank

Jar files# file:///usr/lib/spark/examples/jars/spark-examples.jar

Arguments# /data.txt

Max restarts per hour# 1

Click on Submit

Task 3: Run a simple Dataprep job

1. Select Dataprep from the Main Navigation Menu and open it in a new tab
2. Welcome to Cloud Dataprep, a partner collaboration

Select Checkbox Terms of Service

Click on Accept

3. Allow Trifacta to access project data

Select Checkbox and click on Agree and Continue

Click on Allow

4. Click on Continue of the pop-up box

5. On the right side of the screen, click on Import Data

6. On the new page, Click on Cloud Storage

7. Choose a file or folder, click on pencil sign beside Cloud storage

8. Paste the following path

gs://cloud-training/gsp323/runs.csv

9. Once the dataset is auto-populated, at the bottom click on Continue

10. Click on runs.csv

11. On the right side top click on Use in new flow

12. Select Column 10

Select Failure

Delete rows with selected values

13. Select Column 9

Right click <Filter Rows<on Column values <contains

Pattern to match

 /(⁰$|⁰\.0$)/

Keep matching rows

Click Add

14. Rename all the columns name

Task 4. AI

1. open the cloud shell and run below commands

gcloud iam service-accounts create my-natlang-sa \
 — display-name “my natural language service account”

gcloud iam service-accounts keys create ~/key.json \

 — iam-account my-natlang-sa@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com

export GOOGLE_APPLICATION_CREDENTIALS=”/home/$USER/key.json”

gcloud auth activate-service-account my-natlang-sa@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com — key-file=$GOOGLE_APPLICATION_CREDENTIALS

gcloud ml language analyze-entities — content=”Old Norse texts portray Odin as one-eyed and long-bearded, frequently wielding a spear named Gungnir and wearing a cloak and a broad hat.” > result.json

gcloud auth login — no-launch-browser

1. After running above command it will output a url in cloud please click on the link it will open in new page select username > click on allow & copy the verification code from there and return back to cloud shell and paste there
2. Run below command by replacing the the path with the PATH
3. On the lab instruction page Task 4 point no. 1 you can see the highlighted path in yellow color please replace that with the path in below command.

gsutil cp result.json PATH

1. Go to the API & Services > credentials from navigation menu and click on +create credentials > API Key (note down your API key on notepad)
2. Run below command by replacing API Key which is noted down

export API_KEY={Replace with API KEY}

1. Create request.json file using command : nano request.json
2. Paste below code in request.json file and click on ctrl + x, then press y, and hit enter to save the file

{
“config”: {
“encoding”:”FLAC”,
“languageCode”: “en-US”
},

“audio”: {

“uri”:”gs://cloud-training/gsp323/task4.flac”
}
}

1. Run below command for in cloud shell after making request,json file

curl -s -X POST -H “Content-Type: application/json” — data-binary @request.json \
“https://speech.googleapis.com/v1/speech:recognize?key=${API_KEY}" > result.json

Run below command by replacing the the path with the PATH You can find it on 1st

On the lab instruction page Task 4 point no. 1 you can see the highlighted path in yellow color please replace that with the path in below command.

gsutil cp result.json PATH

gcloud iam service-accounts create quickstart

gcloud iam service-accounts keys create key.json — iam-account quickstart@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com

gcloud auth activate-service-account — key-file key.json

export ACCESS_TOKEN=$(gcloud auth print-access-token)