Enable Mutual TLS Authentication for Kafka Clients

01 Sign in to the AWS Management Console.

02 Navigate to Amazon MSK console at https://console.aws.amazon.com/msk.

03 In the main navigation panel, under MSK Clusters, choose Clusters.

04 Click on the name (link) of the cluster that you want to examine, available in the Cluster namecolumn.

05 Select the Properties tab and check the TLS client authentication through AWS Certificate Manager (ACM) attribute value available in the Security settings section. If TLS client authentication through AWS Certificate Manager (ACM) is set to Not enabled, mutual TLS (mTLS) authentication is not enabled for the selected Amazon MSK cluster.

06 Repeat step no. 4 and 5 for each Amazon Managed Streaming for Kafka (MSK) cluster provisioned within the current AWS region.

07 Change the AWS cloud region from the navigation bar and repeat the Audit process for the other regions.

01 Runlist-clusters command (OSX/Linux/UNIX) using custom query filters to list the Amazon Resource Names (ARNs) of the Amazon MSK clusters available in the selected AWS region:

aws kafka list-clusters
  --region us-east-1
  --query 'ClusterInfoList[*].ClusterArn'

02 The command output should return an array with the requested ARNs:

[
	"arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab",
	"arn:aws:kafka:us-east-1:123456789012:cluster/cc-msk-app-cluster/aabbccdd-1234-aabb-1234-aabbccddaabb-cd"
]

03 Rundescribe-cluster command (OSX/Linux/UNIX) using the ARN of the Amazon MSK cluster that you want to examine as the identifier parameter and custom query filters to determine if mutual TLS authentication is enabled for the selected cluster:

aws kafka describe-cluster
  --region us-east-1
  --cluster-arn arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab
  --query 'ClusterInfo.ClientAuthentication.Tls.Enabled'

04 The command output should return the mTLS configuration status (true for enabled, false for disabled):

false

05 Repeat steps no. 3 and 4 for each Amazon Managed Streaming for Kafka (MSK) cluster available in the selected AWS region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 5 to perform the Audit process for other regions.

01 CloudFormation template (JSON):

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Description": "Enable Mutual TLS Authentication for Kafka Clients",
	"Resources": {
		"MSKCluster": {
			"Type": "AWS::MSK::Cluster",
			"Properties": {
				"ClusterName": "cc-production-msk-cluster",
				"KafkaVersion": "3.4.0",
				"NumberOfBrokerNodes": 2,
				"BrokerNodeGroupInfo": {
					"BrokerAZDistribution": "DEFAULT",
					"ClientSubnets": [
						"subnet-0abcd1234abcd1234",
						"subnet-01234abcd1234abcd"
					],
					"InstanceType": "kafka.m5.large",
					"SecurityGroups": [
						"sg-0abcd1234abcd1234"
					],
					"StorageInfo": {
						"EbsStorageInfo": {
							"VolumeSize": 500
						}
					}
				},
				"EncryptionInfo": {
					"EncryptionInTransit": {
						"InCluster": true,
						"ClientBroker": "TLS"
					}
				},
				"ClientAuthentication": {
					"Tls": {
						"Enabled": true,
						"CertificateAuthorityArnList": [
							"arn:aws:acm-pca:us-east-1:123456789012:certificate-authority/abcdabcd-1234-1234-1234-abcdabcdabcd"
						]
					}
				}
			}
		}
	}
}

02 CloudFormation template (YAML):

AWSTemplateFormatVersion: '2010-09-09'
	Description: Enable Mutual TLS Authentication for Kafka Clients
	Resources:
		MSKCluster:
		Type: AWS::MSK::Cluster
		Properties:
			ClusterName: cc-production-msk-cluster
			KafkaVersion: 3.4.0
			NumberOfBrokerNodes: 2
			BrokerNodeGroupInfo:
			BrokerAZDistribution: DEFAULT
			ClientSubnets:
				- subnet-0abcd1234abcd1234
				- subnet-01234abcd1234abcd
			InstanceType: kafka.m5.large
			SecurityGroups:
				- sg-0abcd1234abcd1234
			StorageInfo:
				EbsStorageInfo:
				VolumeSize: 500
			EncryptionInfo:
			EncryptionInTransit:
				InCluster: true
				ClientBroker: TLS
			ClientAuthentication:
			Tls:
				Enabled: true
				CertificateAuthorityArnList:
				- arn:aws:acm-pca:us-east-1:123456789012:certificate-authority/abcdabcd-1234-1234-1234-abcdabcdabcd

01 Terraform configuration file (.tf):

terraform {
	required_providers {
		aws = {
			source  = "hashicorp/aws"
			version = "~> 4.0"
		}
	}

	required_version = ">= 0.14.9"
}

provider "aws" {
	profile = "default"
	region  = "us-east-1"
}

resource "aws_msk_cluster" "msk-cluster" {
	cluster_name           = "cc-production-msk-cluster"
	kafka_version          = "3.4.0"
	number_of_broker_nodes = 2

	broker_node_group_info {
		instance_type  = "kafka.m5.large"
		client_subnets = ["subnet-0abcd1234abcd1234","subnet-01234abcd1234abcd"]
		storage_info {
			ebs_storage_info {
			volume_size = 500
			}
		}
		security_groups = ["sg-0abcd1234abcd1234"]
	}

	encryption_info {
		encryption_in_transit {
			in_cluster    = true
			client_broker = "TLS"
		}
	}

	# Enable Mutual TLS Authentication for Kafka Clients
	client_authentication {
		tls {
			certificate_authority_arns = ["arn:aws:acm-pca:us-east-1:123456789012:certificate-authority/abcdabcd-1234-1234-1234-abcdabcdabcd"]
		}
	}
}

01 Sign in to the AWS Management Console.

02 To use client authentication, you need an ACM Private Certificate Authority (PCA). To create an ACM PCA required for TLS authentication, navigate to AWS Certificate Manager Private Certificate Authority console at https://console.aws.amazon.com/acm-pca/, choose Create a private CA, and perform the following actions:

1. For Step 1 Select CA type, select Root CA to create a root Certificate Authority (CA). Choose Next to continue.
2. For Step 2 Configure CA subject name, name your CA using the distinguished name (DN) format and provide the required information. Choose Next to continue the setup.
3. For Step 3 Configure CA key algorithm, choose RSA 2048 (default)from the Key algorithm options section. Choose Next to continue.
4. (Optional) For Step 4 Configure revocation - optional, choose whether to enable certificate revocation list (CRL) distribution or the Online Certificate Status Protocol (OCSP) to communicate revocation status. Choose Next to continue.
5. (Optional) For Step 5 Add tags – optional, you can attach tags to your new ACM PCA by using the Add new tag button available in the Tags section. Choose Next to continue.
6. (Optional) For Step 6 Configure CA permissions, select the Authorize ACM access to renew certificates requested by this account checkbox to delegate automatic renewal permissions to the ACM service principal. Choose Next to continue.
7. For Step 7 Review and create, confirm that your configuration is correct, then choose Confirm and create to create your new ACM Private Certificate Authority (PCA).
8. Once your ACM PCA is successfully created, choose Cancel to return to the Private Certificate Authority page.

03 Navigate to Amazon MSK console at https://console.aws.amazon.com/msk.

04 In the main navigation panel, under MSK Clusters, choose Clusters.

05 Click on the name (link) of the cluster that you want to reconfigure.

06 Select the Properties tab and choose Edit from the Security settingssection to modify the access control methods available for the selected cluster.

07 In the Security settings configuration section, select the TLS client authentication through AWS Certificate Manager (ACM) checkbox and choose the ACM PCA created at step no. 2 from the ACM Private CAs dropdown list. Choose Save changes to apply the changes. This will enable mutual TLS authentication for the selected Amazon MSK cluster.

08 Repeat steps no. 2 – 7 for each Amazon MSK cluster that you want to reconfigure, available within the current AWS region.

09 Change the AWS cloud region from the navigation bar and repeat the Remediation process for the other regions.

01 To use client authentication, you need an ACM Private Certificate Authority (PCA). Before you can create an ACM PCA, you must define a CA configuration and save it to a JSON file named ca-config.json. The CA configuration file contains the name and bit size of the private key algorithm, the name of the signing algorithm, and the X.500 certificate subject information:

{
	"KeyAlgorithm":"RSA_2048",
	"SigningAlgorithm":"SHA256WITHRSA",
	"Subject":{
		"Country":"US",
		"Organization":"TM Corp",
		"OrganizationalUnit":"IT",
		"State":"WA",
		"Locality":"Seattle",
		"CommonName":"www.domain.com"
	}
}

02 To create an ACM PCA required for mutual TLS authentication, runcreate-certificate-authority command (OSX/Linux/UNIX) using the CA configuration file defined at the previous step (i.e. ca-config.json) as value for the --certificate-authority-configuration parameter:

aws acm-pca create-certificate-authority
  --region us-east-1
  --certificate-authority-configuration file://ca-config.json
  --certificate-authority-type "ROOT"
  --idempotency-token 012341234
  --tags Key=Name,Value=cc-kafka-pca

03 The command output should return**the ARN of your new ACM PCA:

{
	"arn:aws:acm-pca:us-east-1:123456789012:certificate-authority/abcdabcd-1234-1234-1234-abcdabcdabcd"
}

04 Runupdate-security command (OSX/Linux/UNIX) using the ARN of the Amazon MSK cluster that you want to reconfigure as the identifier parameter, to update the security settings for the selected cluster in order to enable mutual TLS (mTLS) authentication. Use the ARN of the ACM PCA returned at the previous step as value for the Tls.CertificateAuthorityArnList configuration property:

aws kafka update-security
  --region us-east-1
  --cluster-arn arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab
  --current-version ABCDABCDABCDAD
  --client-authentication 'Tls={CertificateAuthorityArnList=["arn:aws:acm-pca:us-east-1:123456789012:certificate-authority/abcdabcd-1234-1234-1234-abcdabcdabcd"],Enabled=true}'

05 The output should return the update-security command request metadata:

{
	"ClusterArn": "arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab",
	"ClusterOperationArn": "arn:aws:kafka:us-east-1:123456789012:cluster/cc-kafka-cluster/abcd1234-abcd-1234-abcd-1234abcd1234-ab/123456789012"
}

06 Repeat steps no. 1 – 5 for each Amazon MSK cluster that you want to reconfigure, available in the selected AWS region.

07 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 6 to perform the Remediation process for other regions.