October 03, 2021
0 Comments

Google Cloud Digital Leader Certification Introduction

Length: 90 minutes

Registration fee: $99

Language: English, Japanese

Exam format: Multiple choice and multiple select

October 03, 2021
0 Comments

Cloud Hosting Evolution (Google Cloud Digital Leader Certification Course)

Hosting Evolution

(Google Cloud Digital Leader Certification Course)

Part 1(a)

Types Of Hosting :

1. Dedicated hosting.

2. Virtual Private Server Hosting .

3. Shared Hosting.

4. Cloud Hosting.

March 06, 2021
1 Comment

Build and Optimize Data Warehouses with BigQuery

Build and Optimize Data Warehouses with BigQuery Challenge Lab, GSP340, Qwiklab

Challenge Lab

Qwiklabs [GSP340]

@prakashfoundation YouTube

click the card to watch the video.

Subscribe Immediately.

@codeGarry Github

Don't forget to follow on GitHub

click me

GSP340

Task 1:

Create a table partitioned by date

code (A):

CREATE OR REPLACE TABLE ecommerce.sample 
PARTITION BY date 
OPTIONS ( partition_expiration_days=90,
description="COVID 19 data"
) AS 
SELECT * 
FROM bigquery-public-data.covid19_govt_response.oxford_policy_tracker 
WHERE alpha_3_code != 'USA' AND alpha_3_code != 'GBR'

Task 2: Add new columns to your table

code (B)


ALTER TABLE ecommerce.sample
ADD COLUMN IF NOT EXISTS population INT64,
ADD COLUMN IF NOT EXISTS country_area FLOAT64,
ADD COLUMN IF NOT EXISTS mobility STRUCT<
avg_retail FLOAT64,
avg_grocery FLOAT64,
avg_parks FLOAT64,
avg_transit FLOAT64,
avg_workplace FLOAT64,
avg_residential FLOAT64
>

Task 3: Add country population data to the population column

code (C):

UPDATE `<PROJECT_ID>.ecommerce.sample` count
SET count.population = count1.pop_data_2019
FROM `bigquery-public-data.covid19_ecdc.covid_19_geographic_distribution_worldwide` count1
WHERE count.date = count1.date AND count.alpha_3_code=count1.country_territory_code

Task 4: Add country area data to the country_area column

code (D):

UPDATE `<PROJECT_ID>.ecommerce.sample` count
SET count.country_area = count1.country_area
FROM `bigquery-public-data.census_bureau_international.country_names_area` count1
WHERE count.country_name = count1.country_name

Task 5: Populate the mobility record data

code (E)

UPDATE `<PROJECT_ID>.ecommerce.sample` count
SET count.mobility = STRUCT<
avg_retail FLOAT64, avg_grocery FLOAT64, avg_parks FLOAT64, avg_transit FLOAT64, avg_workplace FLOAT64, avg_residential FLOAT64
>
(count1.avg_retail, count1.avg_grocery, count1.avg_parks, count1.avg_transit, count1.avg_workplace, count1.avg_residential)
FROM ( SELECT country_region, date, 
      AVG(retail_and_recreation_percent_change_from_baseline) as avg_retail,
      AVG(grocery_and_pharmacy_percent_change_from_baseline)  as avg_grocery,
      AVG(parks_percent_change_from_baseline) as avg_parks,
      AVG(transit_stations_percent_change_from_baseline) as avg_transit,
      AVG( workplaces_percent_change_from_baseline ) as avg_workplace,
      AVG( residential_percent_change_from_baseline)  as avg_residential
      FROM `bigquery-public-data.covid19_google_mobility.mobility_report`
      GROUP BY country_region, date) AS count1
WHERE count.country_name = count1.country_region
AND count.date = count1.date

Task 6: Query missing data in population & country_area columns

code (F):

SELECT DISTINCT country_name
FROM `<			>.ecommerce.sample`
WHERE population is NULL 
UNION ALL
SELECT DISTINCT country_name
FROM `<			>.ecommerce.sample`
WHERE country_area IS NULL 
ORDER BY country_name ASC

March 06, 2021
1 Comment

Ensure-Access-Identity-in-Google-Cloud-Challenge-Lab

Challenge Lab

Qwiklabs [GSP342]

@prakashfoundation YouTube

click the card to watch the video.

Subscribe Immediately.

@codeGarry Github

Don't forget to follow on GitHub

click me

GSP342

Task 1:

Create a custom security role.

Execute this command on cloud shell to create yaml file for custom role:

code (A):

nano role-definition.yaml

Replace the below content in the file:

code (B)

title: "orca_storage_update"
description: "Permissions"
stage: "ALPHA"
includedPermissions:
- storage.buckets.get
- storage.objects.get
- storage.objects.list
- storage.objects.update
- storage.objects.create

Ctrl+x --> Y--> Enter

code (C):

gcloud iam roles create orca_storage_update --project $DEVSHELL_PROJECT_ID \
--file role-definition.yaml

commands for task2 & task3:

Another Method in video if you get error:

code (D):

gcloud iam service-accounts create orca-private-cluster-sa --display-name "my service account"

gcloud alpha projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member serviceAccount:orca-private-cluster-sa@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com --role "projects/<projectid>/roles/orca_storage_update"

gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member serviceAccount:orca-private-cluster-sa@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com --role “roles/monitoring.viewer”

gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member serviceAccount:orca-private-cluster-sa@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com --role “roles/logging.logWriter” 

gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID --member serviceAccount:orca-private-cluster-sa@$DEVSHELL_PROJECT_ID.iam.gserviceaccount.com --role “roles/monitoring.metricWriter”

Task 4: Create and configure a new Kubernetes Engine private cluster

code (E)

gcloud container clusters create orca-test-cluster --num-nodes 1 --master-ipv4-cidr=172.16.0.64/28 --network orca-build-vpc --subnetwork orca-build-subnet --enable-master-authorized-networks  --master-authorized-networks 192.168.10.2/32 --enable-ip-alias --enable-private-nodes --enable-private-endpoint --service-account orca-private-cluster-sa@<Project ID>.iam.gserviceaccount.com --zone us-east1-b

Task 5: Deploy an application to a private Kubernetes Engine cluster.

code 5(a):

gcloud container clusters get-credentials orca-test-cluster --internal-ip --zone us-east1-b --project <Project ID>

code 5(b):

kubectl create deployment hello-server --image=gcr.io/google-samples/hello-app:1.0

March 06, 2021
2 Comments

Insights from Data with BigQuery

Challenge Lab

Qwiklabs [GSP787]

@prakashfoundation YouTube

click the card to watch the video.

Subscribe Immediately.

Task 1:

Total Confirmed Cases

Build a query that will answer "What was the total count of confirmed cases on Apr 15, 2020?" The query needs to return a single row containing the sum of confirmed cases across all countries. The name of the column should be total_cases_worldwide.

code (A):

SELECT sum(cumulative_confirmed) as total_cases_worldwide FROM `bigquery-public-data.covid19_open_data.covid19_open_data` where date='2020-04-15'

Query 2: Worst Affected Areas

Build a query for answering "How many states in the US had more than 100 deaths on Apr 10, 2020?" The query needs to list the output in the field count_of_states. Hint: Don't include NULL values.

code (B)

with deaths_by_states as (
SELECT subregion1_name as state, sum(cumulative_deceased) as death_count
FROM `bigquery-public-data.covid19_open_data.covid19_open_data` 
where country_name="United States of America" and date='2020-04-10' and subregion1_name is NOT NULL
group by subregion1_name
)

select count(*) as count_of_states
from deaths_by_states
where death_count > 100

Query 3: Identifying Hotspots

Build a query that will answer "List all the states in the United States of America that had more than 1000 confirmed cases on Apr 10, 2020?" The query needs to return the State Name and the corresponding confirmed cases arranged in descending order. Name of the fields to return state and total_confirmed_cases.

code (C):

SELECT * FROM (
SELECT subregion1_name as state, sum(cumulative_confirmed) as total_confirmed_cases
FROM `bigquery-public-data.covid19_open_data.covid19_open_data`
WHERE country_code="US" AND date='2020-04-10' AND subregion1_name is NOT NULL
GROUP BY subregion1_name
ORDER BY total_confirmed_cases DESC ) WHERE total_confirmed_cases > 1000

Query 4: Fatality Ratio

Build a query that will answer "What was the case-fatality ratio in Italy for the month of April 2020?" Case-fatality ratio here is defined as (total deaths / total confirmed cases) * 100. Write a query to return the ratio for the month of April 2020 and containing the following fields in the output: total_confirmed_cases, total_deaths, case_fatality_ratio.

code (D):

SELECT sum(cumulative_confirmed) as total_confirmed_cases, sum(cumulative_deceased) as total_deaths, (sum(cumulative_deceased)/sum(cumulative_confirmed))*100 as case_fatality_ratio 
FROM `bigquery-public-data.covid19_open_data.covid19_open_data` 
where country_name="Italy" AND date BETWEEN '2020-04-01'and '2020-04-30'

Query 5: Identifying specific day

Build a query that will answer: "On what day did the total number of deaths cross 10000 in Italy?" The query should return the date in the format yyyy-mm-dd.

code (E)

SELECT date
FROM `bigquery-public-data.covid19_open_data.covid19_open_data` 
where country_name="Italy" and cumulative_deceased>10000
order by date asc
limit 1

Query 6: Finding days with zero net new cases

The following query is written to identify the number of days in India between 21 Feb 2020 and 15 March 2020 when there were zero increases in the number of confirmed cases. However it is not executing properly.

code (F):

WITH india_cases_by_date AS (
  SELECT
    date,
    SUM( cumulative_confirmed ) AS cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name ="India"
    AND date between '2020-02-21' and '2020-03-15'
  GROUP BY
    date
  ORDER BY
    date ASC 
 )
 
, india_previous_day_comparison AS 
(SELECT
  date,
  cases,
  LAG(cases) OVER(ORDER BY date) AS previous_day,
  cases - LAG(cases) OVER(ORDER BY date) AS net_new_cases
FROM india_cases_by_date
)

select count(*)
from india_previous_day_comparison
where net_new_cases=0

Query 7: Doubling rate

Using the previous query as a template, write a query to find out the dates on which the confirmed cases increased by more than 10% compared to the previous day (indicating doubling rate of ~ 7 days) in the US between the dates March 22, 2020 and April 20, 2020. The query needs to return the list of dates, the confirmed cases on that day, the confirmed cases the previous day, and the percentage increase in cases between the days. Use the following names for the returned fields: Date, Confirmed_Cases_On_Day, Confirmed_Cases_Previous_Day and Percentage_Increase_In_Cases.

code (G):

WITH us_cases_by_date AS (
  SELECT
    date,
    SUM(cumulative_confirmed) AS cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name="United States of America"
    AND date between '2020-03-22' and '2020-04-20'
  GROUP BY
    date
  ORDER BY
    date ASC 
 )
 
, us_previous_day_comparison AS 
(SELECT
  date,
  cases,
  LAG(cases) OVER(ORDER BY date) AS previous_day,
  cases - LAG(cases) OVER(ORDER BY date) AS net_new_cases,
  (cases - LAG(cases) OVER(ORDER BY date))*100/LAG(cases) OVER(ORDER BY date) AS percentage_increase
FROM us_cases_by_date
)

select Date, cases as Confirmed_Cases_On_Day, previous_day as Confirmed_Cases_Previous_Day, percentage_increase as Percentage_Increase_In_Cases
from us_previous_day_comparison
where percentage_increase > 10

Query 8: Recovery rate

Build a query to list the recovery rates of countries arranged in descending order (limit to 10) upto the date May 10, 2020. Restrict the query to only those countries having more than 50K confirmed cases. The query needs to return the following fields: country, recovered_cases, confirmed_cases, recovery_rate.

code (H):

WITH cases_by_country AS (
  SELECT
    country_name AS country,
    sum(cumulative_confirmed) AS cases,
    sum(cumulative_recovered) AS recovered_cases
  FROM
    bigquery-public-data.covid19_open_data.covid19_open_data
  WHERE
    date = '2020-05-10'
  GROUP BY
    country_name
 )
 
, recovered_rate AS 
(SELECT
  country, cases, recovered_cases,
  (recovered_cases * 100)/cases AS recovery_rate
FROM cases_by_country
)

SELECT country, cases AS confirmed_cases, recovered_cases, recovery_rate
FROM recovered_rate
WHERE cases > 50000
ORDER BY recovery_rate desc
LIMIT 10

Query 9: CDGR - Cumulative Daily Growth Rate

The following query is trying to calculate the CDGR on May 10, 2020(Cumulative Daily Growth Rate) for France since the day the first case was reported. The first case was reported on Jan 24, 2020.

code (I):

WITH
  france_cases AS (
  SELECT
    date,
    SUM(cumulative_confirmed) AS total_cases
  FROM
    `bigquery-public-data.covid19_open_data.covid19_open_data`
  WHERE
    country_name="France"
    AND date IN ('2020-01-24',
      '2020-05-10')
  GROUP BY
    date
  ORDER BY
    date)
, summary as (
SELECT
  total_cases AS first_day_cases,
  LEAD(total_cases) OVER(ORDER BY date) AS last_day_cases,
  DATE_DIFF(LEAD(date) OVER(ORDER BY date),date, day) AS days_diff
FROM
  france_cases
LIMIT 1
)
select first_day_cases, last_day_cases, days_diff, POW((last_day_cases/first_day_cases),(1/days_diff))-1 as cdgr
from summary

Create a Datastudio report

Create a Datastudio report that plots the following for the United States

code (J):

SELECT
  date, SUM(cumulative_confirmed) AS country_cases,
  SUM(cumulative_deceased) AS country_deaths
FROM
  `bigquery-public-data.covid19_open_data.covid19_open_data`
WHERE
  date BETWEEN '2020-03-15'
  AND '2020-04-30'
  AND country_name ="United States of America"
GROUP BY date

Google Cloud Digital Leader Certification Introduction

Cloud Hosting Evolution (Google Cloud Digital Leader Certification Course)

Types Of Hosting :

Build and Optimize Data Warehouses with BigQuery

Challenge Lab

Qwiklabs [GSP340]

@prakashfoundation YouTube

@codeGarry Github

GSP340

Task 1:

Task 2: Add new columns to your table

Task 3: Add country population data to the population column

Task 4: Add country area data to the country_area column

Task 5: Populate the mobility record data

Task 6: Query missing data in population & country_area columns

Ensure-Access-Identity-in-Google-Cloud-Challenge-Lab

Challenge Lab

Qwiklabs [GSP342]

@prakashfoundation YouTube

@codeGarry Github

GSP342

Task 1:

Replace the below content in the file:

Ctrl+x --> Y--> Enter

commands for task2 & task3:

Another Method in video if you get error:

Task 4: Create and configure a new Kubernetes Engine private cluster

Task 5: Deploy an application to a private Kubernetes Engine cluster.

Insights from Data with BigQuery

Challenge Lab

Qwiklabs [GSP787]

@prakashfoundation YouTube

Task 1:

Query 2: Worst Affected Areas

Query 3: Identifying Hotspots

Query 4: Fatality Ratio

Query 5: Identifying specific day

Query 6: Finding days with zero net new cases

Query 7: Doubling rate

Query 8: Recovery rate

Query 9: CDGR - Cumulative Daily Growth Rate

Create a Datastudio report

Pages

Courses

Most Famous

Map