Enable GKE Cluster Node Encryption with Customer-Managed Keys

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 Navigate to Google Kubernetes Engine (GKE) console at https://console.cloud.google.com/kubernetes.

04 In the main navigation panel, under Kubernetes Engine, select Clusters to access the list with the GKE clusters provisioned within the selected project.

05 Click on the name (link) of the GKE cluster that you want to examine.

06 Select the NODES tab to access the node pools created for the selected cluster.

07 Click on the name (link) of the cluster node pool that you want to examine.

08 In the Nodes section, check the Boot disk encryption attribute value. If the Boot disk encryption value is set to Google-managed key, the nodes provisioned for the selected cluster node pool are encrypted with a Google-managed key instead of a Customer-Managed Key (CMK).

09 Repeat step no. 7 and 8 for each node pool provisioned for the selected GKE cluster.

10 Repeat steps no. 5 – 9 for each GKE cluster created for the selected GCP project.

11 Repeat steps no. 2 – 10 for each project deployed within your Google Cloud account.

01 Run projects list command (Windows/macOS/Linux) with custom query filters to list the ID of each project available in your Google Cloud account:

gcloud projects list
--format="table(projectId)"

02 The command output should return the requested GCP project ID(s):

PROJECT_ID
cc-bigdata-project-123123
cc-iot-app-project-112233

03 Run container clusters list command (Windows/macOS/Linux) using the ID of the GCP project that you want to examine as the identifier parameter and custom query filters to describe the name and the region of each GKE cluster created for the selected project:

gcloud container clusters list
--project cc-bigdata-project-123123
--format="(NAME,LOCATION)"

04 The command output should return the requested cluster names and their regions:

NAME                     LOCATION
cc-gke-frontend-cluster  us-central1
cc-gke-backend-cluster   us-central1

05 Run container node-pools list command (Windows/macOS/Linux) using the name of the GKE cluster that you want to examine as the identifier parameter, to describe the name of each node pool provisioned for the selected cluster:

gcloud container node-pools list
	--cluster=cc-gke-frontend-cluster
	--region=us-central1
	--format="(NAME)"

06 The command output should return the requested cluster node pool name(s):

NAME
cc-gke-frontend-pool-001
cc-gke-frontend-pool-002

07 Run container node-pools describe command (Windows/macOS/Linux) using the name of the cluster node pool that you want to examine as the identifier parameter and custom output filtering to describe the ID of the Customer-Managed Key (CMK) used to encrypt the node pool data:

gcloud container node-pools describe cc-gke-frontend-pool-001
	--cluster=cc-gke-frontend-cluster
	--region=us-central1
	--format="json(config.bootDiskKmsKey)"

08 The command output should return the full resource ID of the CMK used to encrypt the cluster node pool data:

null

09 Repeat steps no. 7 and 8 for each node pool provisioned for the selected GKE cluster.

10 Repeat steps no. 5 – 9 for each GKE cluster created for the selected GCP project.

11 Repeat steps no. 3 – 10 for each GCP project deployed in your Google Cloud account.

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 Navigate to Google Kubernetes Engine (GKE) console at https://console.cloud.google.com/kubernetes.

04 In the main navigation panel, under Kubernetes Engine, select Clusters to access the list with the GKE Autopilot clusters available within the selected project. A GKE Autopilot cluster has the Mode set to Autopilot.

05 Click on the name (link) of the GKE Autopilot cluster that you want to examine.

06 Select the DETAILS tab to view the configuration information available for the selected cluster.

07 In the Security section, check the Boot disk encryption attribute value. If the Boot disk encryption value is set to Google-managed, the nodes provisioned for the selected GKE Autopilot cluster are encrypted with a Google-managed key instead of a Customer-Managed Key (CMK).

08 Repeat steps no. 5 – 7 for each GKE Autopilot cluster created for the selected GCP project.

09 Repeat steps no. 2 – 8 for each project deployed within your Google Cloud account.

01 Run projects list command (Windows/macOS/Linux) with custom query filters to list the ID of each project available in your Google Cloud account:

gcloud projects list
--format="table(projectId)"

02 The command output should return the requested GCP project ID(s):

PROJECT_ID
cc-bigdata-project-123123
cc-iot-app-project-112233

03 Run container clusters list command (Windows/macOS/Linux) using the ID of the GCP project that you want to examine as the identifier parameter and custom query filters to describe the name and the region of each GKE cluster created for the selected project:

gcloud container clusters list
--project cc-bigdata-project-123123
--format="(NAME,LOCATION)"

04 The command output should return the requested cluster names and their regions:

NAME                     LOCATION
cc-gke-frontend-cluster  us-central1
cc-gke-backend-cluster   us-central1

05 Run container clusters describe command (Windows/macOS/Linux) using the name of the GKE Autopilot cluster that you want to examine as the identifier parameter, to describe the ID of the Customer-Managed Key (CMK) used to encrypt the cluster node pool data:

gcloud container clusters describe cc-gke-frontend-cluster
--region=us-central1
--format="yaml(nodeConfig.bootDiskKmsKey)"

06 The command output should return the full resource ID of the CMK used to encrypt the cluster node pool data:

null

07 Repeat steps no. 5 and 6 for each GKE Autopilot cluster created for the selected GCP project.

08 Repeat steps no. 3 – 7 for each GCP project deployed in your Google Cloud account.

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 To create and configure your own Customer-Managed Key (CMK), perform the following:

1. Before you can set up and manage any Customer-Managed Keys (CMKs), you must create a key ring. A Cloud KMS key ring is a grouping of cryptographic keys made available for organizational purposes in a specific location.
2. Navigate to Cloud KMS console at https://console.cloud.google.com/security/kms.
3. Choose + CREATE KEY RING from the console top menu to create the required key ring for your new Customer-Managed Key (CMK).
4. A key ring requires a name and location. On the Create key ring page, provide a unique name in the Key ring name box, choose the location type, and select the appropriate location from the Location type dropdown list. The location can be either global (Multi-region) or associated with a particular region (Region). If the Customer-Managed Key created later for this key ring will be used to encrypt/decrypt resources in a given region, select that region as the key ring location. Choose CREATE to deploy the new key ring.
5. On the Create key setup page, perform the following:
   • Under What type of key do you want to create?, select Generated key for the type of the CMK that you want to create.
   • Provide a unique name for your new Customer-Managed Key in the Key name box.
   • Choose the protection level that you want to use for the new key from the Protection level dropdown list. Protection level determines how cryptographic operations are performed.
   • Select Symmetric encrypt/decrypt from the Purpose dropdown list to define the types of operations that your cryptographic key can perform.
   • Choose the CMK rotation period from the Key rotation period dropdown list and select the starting date for key rotation.
   • Choose CREATE to deploy your new Customer-Managed Key (CMK).

04 Navigate to Google Kubernetes Engine (GKE) console at https://console.cloud.google.com/kubernetes.

05 In the main navigation panel, under Kubernetes Engine, select Clusters.

06 Click on the name (link) of the GKE cluster that you want to reconfigure.

07 Select the NODES tab to access the node pools created for the selected cluster.

08 Click on the name (link) of the cluster node pool that you want to re-create and collect all the configuration information available for the selected resource.

09 Go back to the NODES tab and choose ADD NODE POOL to initiate the setup process.

10 On the Add a node pool setup page, perform the following actions:

1. For Node pool details, provide the following information:
   • Provide a unique name for the new node pool in the Name box.
   • Enter the number of nodes for the new pool in the Number of nodes box.
   • Choose whether or not to enable cluster auto-scaler. Must match the node pool configuration collected at step no. 8.
   • (Optional) If required, select the Specify node locations checkbox and choose additional zone nodes.
   • (Optional) If required, configure the Surge Upgrade feature for the new node pool. Must match the node pool configuration collected at step no. 8.
2. For Nodes, provide the following information:
   • Select the type of the node image from the Image type dropdown list.
   • Choose the machine family, type, and series for the new node pool. Select the appropriate boot disk type and size. Must match the node pool configuration collected at step no. 8.
   • Select the Enable customer-managed encryption for boot disk checkbox and select the Customer-Managed Key (CMK) created at step no. 3 from the Select a customer-managed key dropdown list. If your new CMK does not appear in the dropdown list, select DON'T SEE YOUR KEY? ENTER KEY RESOURCE NAME and provide the full resource ID of your CMK. In the "The service-<project-number>@compute-system.iam.gserviceaccount.com service account does not have the "cloudkms.cryptoKeyEncrypterDecrypter" role. Verify the service account has permission to encrypt/decrypt with the selected key" configuration box, choose GRANT to grant the specified service account the required IAM role on the selected CMK.
   • Enter the maximum number of Kubernetes Pods per node in the Maximum Pods per node box.
3. For Security, provide the following information:
   • Choose the service account required by the cluster node pool from the Service account dropdown list.
   • Select the appropriate access scope(s). Must match the node pool configuration collected at step no. 8.
   • Under Shielded options, choose whether to enable integrity monitoring and/or secure boot. Must match the node pool configuration collected at step no. 8.
4. For Metadata, add any required resource labels (tags), and configure the metadata settings such as GCE instance metadata based on the configuration information taken from the source node pool at step no. 8.
5. Choose CREATE to create the new, encrypted node pool.

11 (Optional) Once the new cluster node pool is operating successfully, you can remove the source node pool in order to stop adding charges to your Google Cloud bill. Go back to the NODES tab and perform the following actions:

1. Click on the name (link) of the source node pool that you want to delete.
2. Choose DELETE from the console top menu to initiate the removal process.
3. In the confirmation box, choose DELETE to confirm the node pool deletion.

12 Repeat steps no. 8 – 11 to enable encryption with Customer-Managed Keys for other node pools provisioned within the selected GKE cluster.

13 Repeat step no. 6 – 12 for each GKE cluster that you want to reconfigure, created for the selected GCP project.

14 Repeat steps no. 2 – 13 for each GCP project available in your Google Cloud account.

01 Before you can set up and manage your Customer-Managed Key (CMK), you must create a key ring to store the CMK. Run kms keyrings create command (Windows/macOS/Linux) to create a new Cloud KMS key ring in the specified location. If the CMK created later for this key ring will be used to encrypt/decrypt resources in a given region, select that region as the key ring location:

gcloud kms keyrings create cc-cloud-gke-key-ring
--location=us
--project=cc-bigdata-project-123123
--format="table(name)"

02 The command output should return the identifier (name) of the new key ring:

NAME
projects/cc-bigdata-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring

03 Run kms keys create command (Windows/macOS/Linux) to create a new Customer-Managed Key (CMK) within the Cloud KMS key ring created at the previous steps:

gcloud kms keys create cc-cloud-gke-cmk
--location=us-central1
--keyring=cc-cloud-gke-key-ring
--purpose=encryption
--protection-level=software
--rotation-period=90d
--next-rotation-time=2020-7-15T15:00:00.0000Z
--format="table(name)"

04 The command output should return the full name of the new Customer-Managed Key (CMK):

NAME
projects/cc-bigdata-project-123123/locations/us-central1/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-cmk

05 Run projects add-iam-policy-binding command (Windows/macOS/Linux) to assign the "CryptoKey Encrypter/Decrypter" role to the appropriate service account. Replace <kms-project-id> with the ID of the Google Cloud project where the Customer-Managed Key has been created, and replace <project-number> with the number of the Google Cloud project that is running your GKE cluster:

gcloud projects add-iam-policy-binding <kms-project-id>
--member serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com
--role roles/cloudkms.cryptoKeyEncrypterDecrypter

06 The command output should return the updated IAM policy (YAML format):

Updated IAM policy for project <kms-project-id>.
bindings:
- members:
  - serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com
  role: roles/cloudkms.cryptoKeyEncrypterDecrypter
- members:
  - user:admin@cloudconformity.com
  role: roles/owner
etag: abcdabcdabcd
version: 1

07 Run container node-pools describe command (Windows/macOS/Linux) using the name of the cluster node pool that you want to re-create as the identifier parameter and custom output filtering to describe the configuration information available for the selected node pool:

gcloud container node-pools describe cc-gke-frontend-pool-001
--cluster=cc-gke-frontend-cluster
--region=us-central1
--format=json

08 The command output should return the requested configuration information:

{
  "config": {
    "diskSizeGb": 150,
    "diskType": "pd-standard",
    "imageType": "COS",
    "metadata": {
      "disable-legacy-endpoints": "true"
    },
    "serviceAccount": "default",
    "shieldedInstanceConfig": {
      "enableIntegrityMonitoring": true
    }
  },
  "locations": [
    "us-central1-a",
    "us-central1-b"
  ],

  ...

  "management": {
    "autoRepair": true,
    "autoUpgrade": true
  },
  "maxPodsConstraint": {
    "maxPodsPerNode": "110"
  },
  "name": "cc-gke-frontend-pool-001",
  "podIpv4CidrSize": 24,
  "status": "RUNNING",
  "upgradeSettings": {
    "maxSurge": 1
  },
  "version": "1.14.10-gke.36"
}

09 Run container node-pools create command (Windows/macOS/Linux) using the information returned at the previous step as the configuration data for the command parameters, to create a new GKE cluster node pool, encrypted with the Customer-Managed Key (CMK) created at step no. 3:

gcloud beta container node-pools create cc-gke-encrypted-pool-001
--cluster=cc-gke-frontend-cluster
--region=us-central1
--disk-type=pd-standard
--disk-size=150
--boot-disk-kms-key=projects/cc-bigdata-project-123123/locations/us-central1/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-cmk

10 The command output should return the full URL of the new cluster node pool:

Created [https://dataproc.googleapis.com/v1/projects/cc-bigdata-project-123123/regions/us-central1/clusters/cc-encrypted-dataproc-cluster]

11 (Optional) Once the new node pool is operating successfully, you can remove the source node pool in order to stop adding charges to your GCP bill. Run container node-pools delete command (Windows/macOS/Linux) using the name of the resource that you want to remove as the identifier parameter, to remove the specified node pool from your GKE cluster:

gcloud container node-pools delete cc-gke-frontend-pool-001
--cluster=cc-gke-frontend-cluster
--region=us-central1

12 Type Y to confirm the cluster node pool removal:

The following node pool will be deleted.
[cc-gke-frontend-pool-001] in cluster [cc-gke-frontend-cluster] in [us-central1]
Do you want to continue (Y/n)?  Y

13 The output should return the container node-pools delete command request status:

Deleting node pool cc-gke-frontend-pool-001...done.
Deleted [https://container.googleapis.com/v1/projects/cc-bigdata-project-123123/zones/us-central1/clusters/cc-gke-frontend-cluster/nodePools/cc-gke-frontend-pool-001].

14 Repeat steps no. 7 – 13 to enable encryption with Customer-Managed Keys for other node pools provisioned in the selected GKE cluster.

15 Repeat step no. 7 – 14 for each GKE cluster that you want to reconfigure, created for the selected GCP project.

16 Repeat steps no. 1 – 15 for each GCP project deployed in your Google Cloud account.

01 Sign in to the Google Cloud Management Console.

02 Select the GCP project that you want to examine from the console top navigation bar.

03 To create and configure your own Customer-Managed Key (CMK), perform the following:

1. Before you can set up and manage any Customer-Managed Keys (CMKs), you must create a key ring. A Cloud KMS key ring is a grouping of cryptographic keys made available for organizational purposes in a specific location.
2. Navigate to Cloud KMS console at https://console.cloud.google.com/security/kms.
3. Choose + CREATE KEY RING from the console top menu to create the required key ring for your new Customer-Managed Key (CMK).
4. A key ring requires a name and location. On the Create key ring page, provide a unique name in the Key ring name box, choose the location type, and select the appropriate location from the Location type dropdown list. The location can be either global (Multi-region) or associated with a particular region (Region). If the Customer-Managed Key created later for this key ring will be used to encrypt/decrypt resources in a given region, select that region as the key ring location. Choose CREATE to deploy the new key ring.
5. On the Create key setup page, perform the following:
   • Under What type of key do you want to create?, select Generated key for the type of the CMK that you want to create.
   • Provide a unique name for your new Customer-Managed Key in the Key name box.
   • Choose the protection level that you want to use for the new key from the Protection level dropdown list. Protection level determines how cryptographic operations are performed.
   • Select Symmetric encrypt/decrypt from the Purpose dropdown list to define the types of operations that your cryptographic key can perform.
   • Choose the CMK rotation period from the Key rotation period dropdown list and select the starting date for key rotation.
   • Choose CREATE to deploy your new Customer-Managed Key (CMK).

04 Navigate to Google Kubernetes Engine (GKE) console at https://console.cloud.google.com/kubernetes.

05 In the main navigation panel, under Kubernetes Engine, select Clusters.

06 Click on the name (link) of the GKE Autopilot cluster that you want to re-create. A GKE Autopilot cluster has the Mode set to Autopilot.

07 Select the DETAILS tab and collect all the configuration information available for the selected cluster.

08 Go back to the Clusters page and choose CREATE to initiate the cluster setup process.

09 On the Create cluster selection page, choose CONFIGURE next to GKE Autopilot to create an Autopilot cluster.

10 For Region, choose the regional location in which your cluster's control plane and nodes will be located.

11 For Network access, choose the type of network you want to allow to access your GKE cluster's workloads.

12 Click on the NETWORKING OPTIONS tab and configure the network settings available for your cluster. Must match the network configuration collected at step no. 7.

13 Click on the ADVANCED OPTIONS tab and perform the following actions:

1. Select the appropriate release channel from the Release Channel dropdown list.
2. Under Automation, configure the cluster maintenance window.
3. Under Anthos Service Mesh, choose whether or not to enable Anthos Service Mesh.
4. Under Automation, perform the following:
   • Choose whether to enable Google Groups for RBAC. Google Groups for RBAC allows you to grant roles to all members of a Google Workspace group. Must match the cluster configuration collected at step no. 7.
   • Select the Encrypt secrets at the application layer checkbox to enable encryption of Kubernetes secrets using Customer-Managed Keys (CMKs).
   • Select the Enable customer-managed encryption for boot disk checkbox and select the Customer-Managed Key (CMK) created at step no. 3 from the Select a customer-managed key dropdown list. If your new CMK does not appear in the dropdown list, select DON'T SEE YOUR KEY? ENTER KEY RESOURCE NAME and provide the full resource ID of your CMK. In the "The service-<project-number>@compute-system.iam.gserviceaccount.com service account does not have the "cloudkms.cryptoKeyEncrypterDecrypter" role. Verify the service account has permission to encrypt/decrypt with the selected key" configuration box, choose GRANT to grant the specified service account the required IAM role on the selected CMK.
5. For Metadata, provide a description for your new GKE cluster.
6. For Labels, add any required resource labels (tags) in order to organize your cluster.

14 Choose CREATE to create the new, encrypted GKE Autopilot cluster.

15 (Optional) Once the new cluster is operating successfully, you can remove the source cluster in order to stop adding charges to your Google Cloud bill. Go back to the Clusters page and perform the following actions:

1. Select the GKE Autopilot cluster that you want to delete.
2. Choose DELETE from the console top menu to initiate the removal process.
3. In the confirmation box, confirm deletion by typing the cluster name, and choose DELETE to terminate the selected cluster.

16 Repeat step no. 6 – 15 for each GKE cluster that you want to re-create, provisioned for the selected GCP project.

17 Repeat steps no. 2 – 16 for each GCP project available in your Google Cloud account.

01 Before you can set up and manage your Customer-Managed Key (CMK), you must create a key ring to store the CMK. Run kms keyrings create command (Windows/macOS/Linux) to create a new Cloud KMS key ring in the specified location. If the CMK created later for this key ring will be used to encrypt/decrypt resources in a given region, select that region as the key ring location:

gcloud kms keyrings create cc-cloud-gke-key-ring
--location=us
--project=cc-bigdata-project-123123
--format="table(name)"

02 The command output should return the identifier (name) of the new key ring:

NAME
projects/cc-bigdata-project-123123/locations/us/keyRings/cc-cloud-gke-key-ring

03 Run kms keys create command (Windows/macOS/Linux) to create a new Customer-Managed Key (CMK) within the Cloud KMS key ring created at the previous steps:

gcloud kms keys create cc-cloud-gke-cmk
--location=us-central1
--keyring=cc-cloud-gke-key-ring
--purpose=encryption
--protection-level=software
--rotation-period=90d
--next-rotation-time=2020-7-15T15:00:00.0000Z
--format="table(name)"

04 The command output should return the full name of the new Customer-Managed Key (CMK):

NAME
projects/cc-bigdata-project-123123/locations/us-central1/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-cmk

05 Run projects add-iam-policy-binding command (Windows/macOS/Linux) to assign the "CryptoKey Encrypter/Decrypter" role to the appropriate service account. Replace <kms-project-id> with the ID of the Google Cloud project where the Customer-Managed Key has been created, and replace <project-number> with the number of the Google Cloud project that is running your GKE cluster:

gcloud projects add-iam-policy-binding <kms-project-id>
--member serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com
--role roles/cloudkms.cryptoKeyEncrypterDecrypter

06 The command output should return the updated IAM policy (YAML format):

Updated IAM policy for project <kms-project-id>.
bindings:
- members:
  - serviceAccount:service-<project-number>@compute-system.iam.gserviceaccount.com
  role: roles/cloudkms.cryptoKeyEncrypterDecrypter
- members:
  - user:admin@cloudconformity.com
  role: roles/owner
etag: abcdabcdabcd
version: 1

07 Run container clusters describe command (Windows/macOS/Linux) using the name of the GKE Autopilot cluster that you want to re-create as the identifier parameter and custom output filtering to describe the configuration information available for the selected cluster:

gcloud container clusters describe cc-gke-frontend-cluster
--region=us-central1
--format="json"

08 The command output should return the requested configuration information (JSON format):

{
  "addonsConfig": {
    "dnsCacheConfig": {
      "enabled": true
    },
    "gcePersistentDiskCsiDriverConfig": {
      "enabled": true
    },
    "gcpFilestoreCsiDriverConfig": {
      "enabled": true
    },
    "networkPolicyConfig": {
      "disabled": true
    }
  },
  "autopilot": {
    "enabled": true
  },

  ...

  "releaseChannel": {
    "channel": "REGULAR"
  },
  "shieldedNodes": {
    "enabled": true
  },
  "status": "RUNNING",
  "subnetwork": "default",
  "verticalPodAutoscaling": {
    "enabled": true
  },
  "zone": "us-central1"
}

09 Run container clusters create-auto command (Windows/macOS/Linux) using the information returned at the previous step as the configuration data for the command parameters, to create a new GKE Autopilot cluster, encrypted with the Customer-Managed Key (CMK) created at step no. 3:

gcloud container clusters create-auto cc-gke-new-frontend-cluster
--project cc-bigdata-project-123123
--region us-central1
--release-channel "regular"
--network "projects/cc-bigdata-project-123123/global/networks/default"
--subnetwork "projects/cc-bigdata-project-123123/regions/us-central1/subnetworks/default"
--cluster-ipv4-cidr "/17"
--services-ipv4-cidr "/22"
--boot-disk-kms-key=projects/cc-bigdata-project-123123/locations/us-central1/keyRings/cc-cloud-gke-key-ring/cryptoKeys/cc-cloud-gke-cmk

10 The command output should return the full URL of the new GKE Autopilot cluster:

Created [https://container.googleapis.com/v1/projects/cc-bigdata-project-123123/zones/us-central1/clusters/cc-gke-new-frontend-cluster]

11 (Optional) Once the new GKE cluster is operating successfully, you can remove the source cluster in order to stop adding charges to your GCP bill. Run container clusters delete command (Windows/macOS/Linux) using the name of the resource that you want to remove as the identifier parameter, to remove the specified GKE cluster:

gcloud container clusters delete cc-gke-frontend-cluster
--region=us-central1

12 Type Y to confirm the cluster removal:

The following clusters will be deleted.
[cc-gke-frontend-cluster] in [us-central1]
Do you want to continue (Y/n)?  Y

13 The output should return the container clusters delete command request status:

Deleting cluster cc-gke-frontend-cluster...done.
Deleted [https://container.googleapis.com/v1/projects/cc-bigdata-project-123123/zones/us-central1/clusters/cc-gke-frontend-cluster].

14 Repeat step no. 7 – 13 for each GKE Autopilot cluster that you want to re-create, provisioned for the selected GCP project.

15 Repeat steps no. 1 – 14 for each GCP project deployed in your Google Cloud account.