- Knowledge Base
- Amazon Web Services
- Amazon Kinesis Data Firehose
- Enable Firehose Delivery Stream Server-Side Encryption
Ensure that your Amazon Kinesis Data Firehose delivery streams are encrypted using Server-Side Encryption. It is recommended for added security to use KMS Customer-managed Customer Master Keys (CMKs) instead of AWS managed-keys, in order to have full control over the encryption and decryption process and meet regulatory requirements. Amazon Kinesis Data Firehose is a fully managed service designed for real-time streaming data delivery to destinations such as Amazon S3, Amazon Redshift, Amazon ElasticSearch Service, and Splunk.
This rule resolution is part of the Conformity Security & Compliance tool for AWS.
When you use your own Amazon KMS Customer-managed CMKs to protect your Kinesis Data Firehose delivery streams, you have complete control over who can use the encryption keys to access your streaming data. Amazon KMS service allows you to easily create, rotate, disable, and audit the CMK encryption keys for your Kinesis Data Firehose delivery streams.
Audit
To determine the encryption status for your Firehose delivery streams, perform the following actions:
Using AWS Console
01 Sign in to the AWS Management Console.
02 Navigate to Amazon Kinesis console at https://console.aws.amazon.com/kinesis/.
03 In the navigation panel, under Amazon Kinesis, choose Delivery streams.
04 Click on the name (link) of the Firehose delivery stream that you want to examine.
05 Select the Configuration tab to access the stream configuration details.
06 In the Server-side encryption (SSE) section, check the Encryption type attribute value. If the Encryption type attribute is not available, the selected Firehose delivery stream is not encrypted using SSE. If the Encryption type attribute value is set to AWS-owned CMK, the selected Amazon Kinesis Data Firehose delivery stream is encrypted using the default master key (AWS-managed key) instead of a KMS Customer-managed CMK.
07 Repeat steps no. 4 - 6 for each Firehose delivery stream available within the current AWS region.
08 Change the AWS cloud region from the navigation bar to repeat the Audit process for other regions.
Using AWS CLI
01 Run list-delivery-streams command (OSX/Linux/UNIX) to describe the name of each Kinesis Data Firehose delivery stream available in the selected AWS region:
aws firehose list-delivery-streams --region us-east-1 --query 'DeliveryStreamNames'
02 The command output should return the requested delivery stream name(s):
[ "cc-project5-delivery-stream", "cc-iot-system-delivery-stream" ]
03 Run describe-delivery-stream command (OSX/Linux/UNIX) using the Firehose delivery stream that you want to examine as the identifier parameter and custom query filters to describe the type of the master key used for encrypting the selected delivery stream:
aws firehose describe-delivery-stream --region us-east-1 --delivery-stream-name cc-project5-delivery-stream --query 'DeliveryStreamDescription.DeliveryStreamEncryptionConfiguration.KeyType'
04 The command output should return the type of the associated master key:
"AWS_OWNED_CMK"
If the describe-delivery-stream command output returns null, the selected Firehose delivery stream is not encrypted using SSE. If the command output returns "AWS_OWNED_CMK", as shown in the example above, the selected Amazon Kinesis Data Firehose delivery stream is encrypted using the default master key (AWS-managed key) instead of a KMS Customer-managed CMK.
05 Repeat steps no. 3 and 4 for each Firehose delivery stream available in the selected AWS region.
06 Change the AWS cloud region by updating the --region command parameter value and repeat the Audit process for other regions.
Remediation / Resolution
To encrypt existing Amazon Kinesis Data Firehose delivery streams using your own Amazon KMS Customer-managed CMKs, perform the following actions:
Using AWS Console
01 Sign in to the AWS Management Console.
02 To create your own customer-managed CMK, navigate to Amazon KMS console at https://console.aws.amazon.com/kms/.
03 In the left navigation panel, click Customer managed keys.
04 Select the appropriate AWS region from the navigation bar (must match the region of your Firehose delivery stream).
05 Click the Create Key button from the console top menu to initiate the setup process.
06 For Step 1 Configure key, choose Symmetric from the Key type section, and select KMS for the Key material origin, available under Advanced options. Click Next to continue.
07 For Step 2 Add labels, provide a unique name (alias) and a short description for your new CMK, then use the Add tag button to create any required tag sets (optional). Click Next to continue the setup process.
08 For Step 3 Define key administrative permissions, choose which IAM users and/or roles can administer your new CMK through the KMS API. You may need to add additional permissions for the users or roles to administer the key from the AWS console. Click Next to continue.
09 For Step 4 Define key usage permissions, within This account section, select which IAM users and/or roles can use the new CMK for cryptographic operations. (Optional) In the Other AWS accounts section, click Add another AWS account and enter an external account ID in order to specify another AWS account that can use this KMS CMK to encrypt and decrypt your Firehose delivery streams. The owners of the external AWS accounts must also provide access to this CMK by creating appropriate policies for their IAM users. Click Next to continue the process.
10 For Step 5 Review and edit key policy, review the key policy, then click Finish to create your new Customer-managed CMK. Once the key is successfully created, the Amazon KMS console will display the following confirmation message: "Success. Your customer master key was created with alias <key-alias> and key ID <key-id>".
11 Once your new Customer-managed CMK is available, navigate to Amazon Kinesis console at https://console.aws.amazon.com/kinesis/.
12 In the navigation panel, under Amazon Kinesis, choose Delivery streams.
13 Click on the name of the Firehose delivery stream that you want to reconfigure.
14 In the Delivery stream details section, check the Source attribute value to determine the data source configured for the selected Firehose delivery stream. Based on the data source available, perform one of the following sets of commands:
- If the Source is set to Direct PUT, perform the following:
- Select the Configuration tab to access the configuration information available for the selected delivery stream.
- In the Server-side encryption (SSE) section, choose Edit.
- Ensure that the Enable server-side encryption for source records in delivery stream checkbox is selected.
- For Encryption type, choose Use customer managed CMK.
- Under Customer managed CMK in KMS, choose Browse, and select the Amazon KMS CMK created at the previous steps.
- Choose Save changes to apply the configuration changes. It takes approximately 30 seconds to enable Server-Side Encryption (SSE) using Customer-managed CMKs.
- If the Source is set to Amazon Kinesis Data Streams, perform the following:
- Select the Configuration tab to access the configuration information available for the selected stream.
- In the Source settings section, click on the name (link) of the associated data stream, available under Kinesis data stream.
- Select the Configuration tab to access the data stream configuration details.
- In the Encryption section, choose Edit.
- Make sure that the Enable server-side encryption checkbox is selected.
- Choose Use customer-managed CMK.
- Under Customer managed CMK in KMS, choose Browse, and select the Amazon KMS CMK created earlier in the Remediation process.
- Choose Save changes to apply the configuration changes.
15 Repeat steps no. 13 and 14 for each Firehose delivery stream that you want to encrypt using Customer Master Keys, available within the current AWS region.
16 Change the AWS cloud region from the navigation bar and repeat the Remediation process for other regions.
Using AWS CLI
01 Define the policy that enables the selected IAM users and/or roles to manage your new Customer-managed CMK, and to encrypt/decrypt your Firehose delivery streams using the KMS API. Create a new policy document (JSON format), name the file data-firehose-cmk-policy.json, and paste the following content (replace the highlighted details, i.e. the ARNs for the IAM users and/or roles, with your own details):
{ "Id": "data-firehose-cmk-policy", "Version": "2012-10-17", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:root
" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "Allow access for Key Administrators", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:role/<role-name>
" }, "Action": [ "kms:Create*", "kms:Describe*", "kms:Enable*", "kms:List*", "kms:Put*", "kms:Update*", "kms:Revoke*", "kms:Disable*", "kms:Get*", "kms:Delete*", "kms:TagResource", "kms:UntagResource", "kms:ScheduleKeyDeletion", "kms:CancelKeyDeletion" ], "Resource": "*" }, { "Sid": "Allow use of the key", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:role/<role-name>
" }, "Action": [ "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" ], "Resource": "*" }, { "Sid": "Allow attachment of persistent resources", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::<aws-account-id>:role/<role-name>
" }, "Action": [ "kms:CreateGrant", "kms:ListGrants", "kms:RevokeGrant" ], "Resource": "*", "Condition": { "Bool": { "kms:GrantIsForAWSResource": "true" } } } ] }
02 Run create-key command (OSX/Linux/UNIX) using the policy document created at the previous step (i.e. data-firehose-cmk-policy.json) as value for the --policy parameter, to create your new customer-provided Customer-managed CMK:
aws kms create-key --region us-east-1 --description 'KMS CMK for encrypting Firehose delivery streams' --policy file://data-firehose-cmk-policy.json --query 'KeyMetadata.Arn'
03 The command output should return the ARN of the new Customer-managed CMK:
"arn:aws:kms:us-east-1:123456789012:key/1234abcd-1234-abcd-1234-abcd1234abcd"
04 Run create-alias command (OSX/Linux/UNIX) using the key ARN returned at the previous step to attach an alias to the new CMK. The alias must start with the prefix "alias/" (the command should not produce an output):
aws kms create-alias --region us-east-1 --alias-name alias/DeliveryStreamCMK --target-key-id arn:aws:kms:us-east-1:123456789012:key/1234abcd-1234-abcd-1234-abcd1234abcd
05 Based on the data source type configured for the selected Firehose delivery stream, perform one of the following sets of commands:
- If the data source is Direct PUT, run start-delivery-stream-encryption command (OSX/Linux/UNIX) using the name of the Firehose delivery stream that you want to reconfigure as the identifier parameter, to enable Server-Side Encryption (SSE) for the selected delivery stream using the KMS Customer-managed CMK created at the previous steps (the command does not produce an output):
aws firehose start-delivery-stream-encryption --region us-east-1 --delivery-stream-name cc-project5-delivery-stream --delivery-stream-encryption-configuration-input KeyARN="arn:aws:kms:us-east-1:123456789012:key/1234abcd-1234-abcd-1234-abcd1234abcd",KeyType="CUSTOMER_MANAGED_CMK"
- If the data source is Amazon Kinesis Data Streams, run start-stream-encryption command (OSX/Linux/UNIX) using the name of the data stream associated with your Firehose delivery stream as the identifier parameter, to enable Server-Side Encryption (SSE) for the specified data stream using the KMS Customer-managed CMK created earlier in the Remediation/Resolution process (the command does not return an output):
aws kinesis start-stream-encryption --region us-east-1 --stream-name cc-project5-data-stream --encryption-type KMS --key-id arn:aws:kms:us-east-1:123456789012:key/1234abcd-1234-abcd-1234-abcd1234abcd
06 Repeat step no. 5 for each Firehose delivery stream that you want to encrypt using Customer Master Keys, available in the selected AWS region.
07 Change the AWS cloud region by updating the --region command parameter value and repeat the Remediation process for other regions.
References
- AWS Documentation
- Amazon Kinesis Data Firehose
- Amazon Kinesis Data Firehose FAQs
- AWS KMS concepts
- Protecting data using server-side encryption with AWS Key Management Service (SSE-KMS)
- AWS Command Line Interface (CLI) Documentation
- firehose
- list-delivery-streams
- describe-delivery-stream
- start-delivery-stream-encryption
- kms
- create-key
- create-alias
- kinesis
- start-stream-encryption