Use Azure Container Networking Interface (CNI) for AKS Clusters

Risk Level: Medium (should be achieved)

Ensure that Azure Kubernetes Service (AKS) clusters are configured to use the Azure Container Networking Interface (CNI) mode instead of the default Kubenet networking mode in order to enhance the segregation of resources and controls in an enterprise environment.

Security

Operational
excellence

Choosing the appropriate network model for your AKS clusters is vital. In production, the Azure Container Networking Interface (CNI) is useful for enabling the segregation of control and resource management. This separation is particularly important for maintaining security, as different teams are often responsible for managing and safeguarding these resources. Azure CNI networking allows direct connectivity to various entities, such as existing Azure resources, on-premises resources, or other services, through individual IP addresses assigned to each pod. By utilizing Azure CNI networking, the virtual network resource resides in a distinct resource group from the AKS cluster. To grant access and administration privileges for these resources, delegate permissions to the AKS cluster identity. The AKS cluster must possess, at a minimum, Network Contributor permissions on the subnet within your virtual network.

Audit

To determine if Azure CNI networking mode is configured for your AKS clusters, perform the following actions:

Using Azure Portal

01 Sign in to the Azure Management Console.

02 Navigate to All resources blade at https://portal.azure.com/#view/HubsExtension/BrowseAll to access all your Microsoft Azure cloud resources.

03 Choose the Azure subscription that you want to access from the Subscription filter box and choose Apply.

04 From the Type filter box, select Kubernetes service and choose Apply to list the Azure Kubernetes Service (AKS) clusters available in the selected subscription.

05 Click on the name (link) of the AKS cluster that you want to examine.

06 In the resource navigation panel, under Settings, select Networking to access the networking settings configured for the selected AKS cluster.

07 Under Network profile, check the Type (plugin) configuration attribute value. If the attribute value is set to Kubenet, the selected Azure Kubernetes Service (AKS) cluster is configured to use the Kubenet networking mode instead of the Azure CNI networking mode.

08 Repeat steps no. 5 – 7 for each AKS cluster provisioned in the selected Azure subscription.

09 Repeat steps no. 3 – 8 for each subscription created in your Microsoft Azure cloud account.

Using Azure CLI

01 Run aks list command (Windows/macOS/Linux) using custom query filters to list the name and the associated resource group for each Azure Kubernetes Service (AKS) cluster available in the current subscription:

az aks list
  --output table
  --query '[*].{name:name, resourceGroup:resourceGroup}'

02 The command output should return the requested AKS cluster names:

Name                     ResourceGroup
----------------------   ------------------------------
cc-project5-aks-cluster  cloud-shell-storage-westeurope
cc-data-mining-cluster   cloud-shell-storage-westeurope

03 Run aks show command (Windows/macOS/Linux) using the name of the AKS cluster that you want to examine and its associated resource group as the identifier parameters to determine the networking mode configured for the selected AKS cluster:

az aks show
  --name cc-project5-aks-cluster
  --resource-group cloud-shell-storage-westeurope
  --query 'networkProfile.networkPlugin'

04 The command output should return the name of the networking plugin used:

"kubenet"

If the aks show command output returns "kubenet", as shown in the example above, the selected Azure Kubernetes Service (AKS) cluster is configured to use the Kubenet networking mode instead of the Azure CNI networking mode.

05 Repeat step no. 3 and 4 for each AKS cluster available within the current Azure subscription.

06 Repeat steps no. 1 – 5 for each subscription created in your Microsoft Azure cloud account.

Remediation / Resolution

To configure Azure CNI networking mode for your AKS clusters, you have to re-create your clusters. To relaunch your AKS clusters with the required networking configuration, perform the following actions:

Using Azure Portal

01 Sign in to the Azure Management Console.

02 Navigate to All resources blade at https://portal.azure.com/#view/HubsExtension/BrowseAll to access all your Microsoft Azure cloud resources.

03 Choose the Azure subscription that you want to access from the Subscription filter box and choose Apply.

04 From the Type filter box, select Kubernetes service and choose Apply to list the Azure Kubernetes Service (AKS) clusters available in the selected subscription.

05 Click on the name (link) of the AKS cluster that you want to re-create and copy the necessary configuration details such as the associated resource group, Azure region, Kubernetes version, node size and count, scaling and networking settings and so on. This configuration information will be required later when the new AKS cluster will be launched.

06 Navigate to Kubernetes Services blade at https://portal.azure.com/#blade/HubsExtension/BrowseResourceBlade/resourceType/Microsoft.ContainerService%2FmanagedClusters.

07 Choose Create and select Create a Kubernetes cluster to initiate the cluster setup wizard.

08 On the Create Kubernetes cluster setup page, configure the AKS new cluster using the configuration information copied at step no. 5:

For Basics, provide a name for the new cluster, select the resource group where the resource will be created, select the appropriate Azure region and Kubernetes version, then configure the cluster node size and count, and choose the appropriate scaling method. Choose Next : Node pools > to continue the setup process.
For Node pools, configure the primary and optional node pools required for the new cluster, then choose Next : Access > to continue the setup.
For Access, configure the authentication and authorization settings required for your AKS cluster. Choose Next : Networking > to continue.
For Networking, choose Azure CNI for Network configuration, select the virtual network and the subnet in which both the nodes and containers within the cluster will be placed, and provide a DNS name prefix to use with the hosted Kubernetes API server FQDN in the DNS name prefix box. Configure the networking security settings required for your cluster, then choose Next : Integrations > to continue.
For Integrations, connect your new AKS cluster with additional Azure cloud services such as Microsoft Defender for Cloud, Azure Container Registry, and Azure Monitor, and configure any required alert rules. Choose Next : Advanced > to continue.
For Advanced, choose whether to use Azure Key Vault as a secrets store with your Kubernetes cluster via a CSI volume, provide the name of the necessary infrastructure resource group, then choose Next : Tags > to continue.
For Tags, create the necessary tags and choose Next : Review + Create > to validate the configuration information provided during setup.
For Review + Create, review the configuration details, then choose Create to launch your new Azure Kubernetes Service (AKS) cluster. The provisioning process should take a few minutes to complete.

09 Repeat steps no. 5 – 8 for each AKS cluster that you want to relaunch, available in the selected subscription.

10 Repeat steps no. 3 – 9 for each subscription created in your Microsoft Azure cloud account.

Using Azure CLI

01 Run aks show command (OSX/Linux/UNIX) using the name of the cluster that you want to re-create as the identifier parameter and custom query filters to get the specified AKS cluster configuration details. The information requested will be required later when the new Azure Kubernetes Service cluster will be launched:

az aks show
  --name cc-project5-aks-cluster
  --resource-group cloud-shell-storage-westeurope

02 The command output should return the requested function configuration information:

{
	"aadProfile": null,
	"addonProfiles": {
		"azureKeyvaultSecretsProvider": {
			"config": null,
			"enabled": false,
			"identity": null
		},
		"azurepolicy": {
			"config": null,
			"enabled": false,
			"identity": null
		}
	},
	"apiServerAccessProfile": null,
	"autoUpgradeProfile": {
		"upgradeChannel": "none"
	},
	"azureMonitorProfile": null,
	"azurePortalFqdn": "cc-project5-aks-cluster-dns-abcdabcd.portal.hcp.westeurope.azmk8s.io",
	"currentKubernetesVersion": "1.25.6",
	"disableLocalAccounts": false,
	"diskEncryptionSetId": null,
	"dnsPrefix": "cc-project5-aks-cluster-dns",
	"enablePodSecurityPolicy": null,
	"enableRbac": true,
	"extendedLocation": null,
	"fqdn": "cc-project5-aks-cluster-dns-abcdabcd.hcp.westeurope.azmk8s.io",
	"fqdnSubdomain": null,
	"httpProxyConfig": null,
	"kubernetesVersion": "1.25.6",
	"linuxProfile": null,
	"location": "westeurope",
	"maxAgentPools": 100,
	"name": "cc-project5-aks-cluster",
	"networkProfile": {
		"dnsServiceIp": "10.0.0.10",
		"ipFamilies": [
			"IPv4"
		],
		"loadBalancerProfile": {
			"enableMultipleStandardLoadBalancers": null,
			"idleTimeoutInMinutes": null,
			"managedOutboundIPs": {
				"count": 1,
				"countIpv6": null
			},
			"outboundIPs": null,
			"outboundIpPrefixes": null
		},
		"loadBalancerSku": "Standard",
		"natGatewayProfile": null,
		"networkDataplane": null,
		"networkMode": null,
		"networkPlugin": "kubenet",
		"networkPluginMode": null,
		"networkPolicy": null,
		"outboundType": "loadBalancer",
		"podCidr": "10.15.0.0/16",
		"podCidrs": [
			"10.15.0.0/16"
		],
		"serviceCidr": "10.0.0.0/16",
		"serviceCidrs": [
			"10.0.0.0/16"
		]
	},
	"nodeResourceGroup": "MC_cloud-shell-storage-westeurope_cc-project5-aks-cluster_westeurope",
	"oidcIssuerProfile": {
		"enabled": false,
		"issuerUrl": null
	},
	"podIdentityProfile": null,
	"powerState": {
		"code": "Running"
	},
	"privateFqdn": null,
	"privateLinkResources": null,
	"provisioningState": "Succeeded",
	"publicNetworkAccess": null,
	"resourceGroup": "cloud-shell-storage-westeurope",
	"securityProfile": {
		"azureKeyVaultKms": null,
		"defender": null,
		"imageCleaner": null,
		"workloadIdentity": null
	},
	"servicePrincipalProfile": {
		"clientId": "msi"
	},
	"sku": {
		"name": "Base",
		"tier": "Free"
	},
	"storageProfile": {
		"blobCsiDriver": null,
		"diskCsiDriver": {
			"enabled": true
		},
		"fileCsiDriver": {
			"enabled": true
		},
		"snapshotController": {
			"enabled": true
		}
	},
	"supportPlan": "KubernetesOfficial",
	"systemData": null,
	"type": "Microsoft.ContainerService/ManagedClusters",
	"windowsProfile": null,
	"workloadAutoScalerProfile": {
		"keda": null
	}
}

03 Run aks create command (OSX/Linux/UNIX) to launch a new Azure Kubernetes Service (AKS) cluster with Azure Container Networking Interface (CNI), using the cluster configuration information returned at step no. 2. For --network-plugin command parameter use azure to configure the new AKS cluster with the Azure CNI networking mode (the command does not produce an output):

az aks create
  --name cc-new-project5-aks-cluster
  --resource-group cloud-shell-storage-westeurope
  --location "westeurope"
  --node-count 3
  --node-vm-size "Standard_B2s"
  --node-osdisk-size 150
  --kubernetes-version "1.25.6"
  --network-plugin azure 
  --vnet-subnet-id /subscriptions/abcdabcd-1234-abcd-1234-abcdabcdabcd/resourceGroups/cloud-shell-storage-westeurope/providers/Microsoft.Network/virtualNetworks/aks-vnet/subnets/aks-subnet
  --dns-service-ip 10.15.0.10
  --service-cidr 10.15.0.0/24
  --generate-ssh-keys

04 Repeat steps no. 1 – 3 for each Azure Kubernetes Service cluster that you want to re-create, available within the current subscription.

05 Repeat steps no. 1 – 5 for each subscription created in your Microsoft Azure cloud account.

References

Azure PowerShell Documentation
az aks list
az aks show
az aks create

Publication date Aug 8, 2023

Unlock the Remediation Steps

Free 30-day Trial

Automatically audit your configurations with Conformity
and gain access to our cloud security platform.

No thanks, back to article

You are auditing:

Use Azure Container Networking Interface (CNI) for AKS Clusters

Risk Level: Medium

Audit

Using Azure Portal

Using Azure CLI

Remediation / Resolution

Using Azure Portal

Using Azure CLI

References

Related AKS rules