Enable BigQuery Dataset Encryption with Customer-Managed Encryption Keys

01 Run projects list command (Windows/macOS/Linux) using custom query filters to list the ID of each Google Cloud Platform (GCP) project available in your cloud account:

gcloud projects list
  --format="table(projectId)"

02 The command output should return the requested GCP project ID(s):

PROJECT_ID
cc-big-data-project-112233
cc-anallytics-project-123123

03 Run bq ls command (using bq Python tool) to list the identifier (name) of each BigQuery dataset created for the specified Google Cloud project:

bq ls
  --project_id cc-ml-project-112233
  --format=pretty

04 The command output should return a table with the requested dataset identifier(s):

+------------------------+
|       datasetId        |
+------------------------+
|   production_dataset   |
|   ad_traffic_dataset   |
+------------------------+

05 Run bq show command (using bq tool) using the ID of the project and the name of the dataset that you want to examine as the identifier parameters, to describe the name of the default Customer-Managed Encryption Key (CMEK) configured for the selected BigQuery dataset:

bq show
  --format=prettyjson cc-big-data-project-112233:production_dataset | jq '.defaultEncryptionConfiguration.kmsKeyName'

06 The command output should return the full name of the Customer-Managed Encryption Key used for dataset encryption:

null

07 Repeat steps no. 5 and 6 for each BigQuery dataset available in the selected GCP project.

08 Repeat steps no. 3 – 7 for each project deployed within your Google Cloud account.

01 To set up and manage Customer-Managed Encryption Keys, you must first create a key ring. A Cloud KMS key ring is a grouping of cryptographic keys made available for organizational purposes in a specific Google Cloud location. Run kms keyrings create command (Windows/macOS/Linux) to create a new Cloud KMS key ring in the specified location (must match the BigQuery dataset location):

gcloud kms keyrings create cc-bigdata-key-ring
  --location=us
  --project=cc-big-data-project-112233
  --format="table(name)"

02 The command output should return the identifier (name) of the newly created key ring:

NAME
projects/cc-big-data-project-112233/locations/us/keyRings/cc-bigdata-key-ring

03 Run kms keys create command (Windows/macOS/Linux) to create a new Cloud KMS Customer-Managed Encryption Key (CMEK) within the KMS key ring provisioned at the previous steps:

gcloud kms keys create cc-bigdata-cmek
  --location=us
  --keyring=cc-bigdata-key-ring
  --purpose=encryption
  --protection-level=software
  --rotation-period=90d
  --next-rotation-time=2022-11-11T12:00:00.0000Z
  --format="table(name)"

04 The command output should return the name of the new Customer-Managed Encryption Key:

NAME
projects/cc-ml-project-112233/locations/us/keyRings/cc-bigdata-key-ring/cryptoKeys/cc-bigdata-cmek

05 Run projects add-iam-policy-binding command (Windows/macOS/Linux) to assign the Cloud KMS "CryptoKey Encrypter/Decrypter" role to the BigQuery system service account. Replace <kms-project-id> with the ID of the Google Cloud project where the Customer-Managed Encryption Key was created, and replace <project-number> with the project number (not the project ID) of the Google Cloud project that is running your BigQuery dataset:

gcloud projects add-iam-policy-binding <kms-project-id>
  --member serviceAccount:bq-<project-number>@bigquery-encryption.iam.gserviceaccount.com
  --role roles/cloudkms.cryptoKeyEncrypterDecrypter

06 The command output should return the updated IAM policy (YAML format):

Updated IAM policy for project <kms-project-id>.
  bindings:
  - members:
    - serviceAccount:bq-<project-number>@bigquery-encryption.iam.gserviceaccount.com
    role: roles/cloudkms.cryptoKeyEncrypterDecrypter
  - members:
    - serviceAccount:bq-<project-number>@bigquery-encryption.iam.gserviceaccount.com
    - user:admin@trendmicro.com
    role: roles/owner
  etag: abcdabcdabcd
  version: 1

07 Run bq update command (using bq tool) to update the selected Google Cloud BigQuery dataset in order to enable encryption at rest using the default Customer-Managed Encryption Key (CMEK) created earlier in the Remediation process, as value for the --default_kms_key command parameter:

bq update
  --default_kms_key projects/cc-ml-project-112233/locations/us/keyRings/cc-bigdata-key-ring/cryptoKeys/cc-bigdata-cmek
  --dataset cc-big-data-project-112233:production_dataset

08 The output should return the bq update command request status:

Dataset 'cc-big-data-project-112233:production_dataset' successfully updated.

09 Repeat steps no. 7 and 8 for each BigQuery dataset available in the selected GCP project.

10 Repeat steps no. 1 – 9 for each project deployed within your Google Cloud account.