Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Enable Real-Time Logging

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)

Ensure that the Real-Time Logging monitoring feature is enabled for your Amazon CloudFront distributions in order to receive log information about requests made to your web content in real time. The real-time logs are delivered to the data stream of your choice in Amazon Kinesis Data Streams. You can build your own Kinesis data stream consumer or use Amazon Kinesis Data Firehose to send the log data to Amazon S3, Amazon Redshift, Amazon ElasticSearch Service, or a third-party log processing service. Amazon CloudFront real-time logs are configurable. You can choose:

  • The sampling rate for your real-time logs, i.e. the percentage of requests for which you want to receive real-time log records.
  • The specific fields that you want to record in real time.
  • The specific cache behaviors (path patterns) that you want to receive real-time logs for.
Operational
excellence

With Amazon CloudFront real-time logs you can get valuable information about requests made to your web content in real time (the log records are delivered within seconds of receiving the requests). You can use these real-time logs to monitor, analyze, and take action based on content delivery performance.


Audit

To determine if your Amazon CloudFront distributions are configured to record real-time logs, perform the following operations:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon CloudFront console at https://console.aws.amazon.com/cloudfront/v3/.

03 In the left navigation panel, under CloudFront, choose Distributions.

04 Click on the name (link) of the active CloudFront distribution that you want to examine. An active distribution has the Status set to Enabled.

05 Select the Behaviors tab to access the cache behavior(s) configured for the selected distribution.

06 Select the cache behavior that you want to examine and choose Edit.

07 On the Edit behavior configuration page, click on Additional settings, and check the Enable real-time logs configuration setting. If the Enable real-time logs setting is set to No, the Real-Time Logging feature is not enabled for the selected cache behavior.

08 Repeat steps no. 6 and 7 to determine the feature status for each cache behavior associated with the selected distribution. If the Real-Time Logging feature is not enabled for the associated cache behavior(s), the selected Amazon CloudFront distribution is not configured to send real-time logs to an Amazon Kinesis data stream.

09 Repeat steps no. 4 – 8 for each active CloudFront distribution available within your AWS cloud account.

Using AWS CLI

01 Run list-distributions command (OSX/Linux/UNIX) with custom query filters to list the ID of each active Amazon CloudFront distribution created in your AWS account:

aws cloudfront list-distributions
  --output table
  --query 'DistributionList.Items[?(Enabled==`true`)].Id'

02 The command output should return a table with the requested distribution ID(s):

--------------------
|ListDistributions |
+------------------+
|  ABCDABCDABCDAB  |
|  AABBCCDDAABBCC  |
+------------------+

03 Run get-distribution-config command (OSX/Linux/UNIX) using the ID of the active Amazon CloudFront distribution that you want to examine as the identifier parameter and custom query filters to describe the Amazon Resource Name (ARN) of the real-time log configuration attached to each custom cache behavior created for the selected distribution. A real-time log configuration contains information about which log fields to record, the sampling rate for log records, and the Kinesis data stream where the real-time logs are delivered:

aws cloudfront get-distribution-config
  --id ABCDABCDABCDAB
  --query 'DistributionConfig.CacheBehaviors.Items[*].RealtimeLogConfigArn'

04 The command output should return the requested configuration information:

[]

If the get-distribution-config command output returns an empty array (i.e. []), as shown in the example above, there are no real-time log configurations available for the associated cache behaviors, therefore the selected Amazon CloudFront distribution is not configured to send real-time logs to an Amazon Kinesis data stream.

05 Repeat steps no. 3 and 4 for each active CloudFront distribution deployed in your AWS cloud account.

Remediation / Resolution

To reconfigure your Amazon CloudFront distributions in order to enable the Real-Time Logging feature, perform the following operations:

Using AWS CloudFormation

01 CloudFormation template (JSON):

{
	"AWSTemplateFormatVersion": "2010-09-09",
	"Parameters": {
		"KinesisStreamArn": {
			"Type": "String"
		},
		"KinesisStreamRoleArn": {
			"Type": "String"
		}
	},
	"Resources": {
		"CloudFrontRealtimeLogConfig": {
			"Type": "AWS::CloudFront::RealtimeLogConfig",
			"Properties": {
				"Name": "cc-real-time-log-config",
				"EndPoints": [
					{
						"KinesisStreamConfig": {
							"RoleArn": {
								"Ref": "KinesisStreamRoleArn"
							},
							"StreamArn": {
								"Ref": "KinesisStreamArn"
							}
						},
						"StreamType": "Kinesis"
					}
				],
				"Fields": [
					"timestamp",
					"c-ip",
					"time-to-first-byte",
					"x-edge-location",
					"x-edge-request-id",
					"x-host-header",
					"x-forwarded-for",
					"ssl-protocol",
					"ssl-cipher"
				],
				"SamplingRate": 100
			}
		},
		"CloudFrontDistribution": {
			"Type": "AWS::CloudFront::Distribution",
			"Properties": {
				"DistributionConfig": {
					"Enabled": true,
					"Comment": "CloudFront CDN Web Distribution",
					"Origins": [
						{
							"Id": "cc-cdn-origin",
							"DomainName": "domain.com",
							"CustomOriginConfig": {
								"HTTPPort": 80,
								"HTTPSPort": 443,
								"OriginKeepaliveTimeout": 15,
								"OriginSSLProtocols": [
									"TLSv1.2"
								],
								"OriginProtocolPolicy": "https-only"
							}
						}
					],
					"DefaultRootObject": "index.html",
					"DefaultCacheBehavior": {
						"Compress": true,
						"AllowedMethods": [
							"HEAD",
							"GET"
						],
						"DefaultTTL": 0,
						"ForwardedValues": {
							"Cookies": {
								"Forward": "none"
							},
							"QueryString": false
						},
						"ViewerProtocolPolicy": "https-only",
						"RealtimeLogConfigArn": {
							"Ref": "CloudFrontRealtimeLogConfig"
						}
					},
					"HttpVersion": "http2",
					"ViewerCertificate": {
						"CloudFrontDefaultCertificate": true
					},
					"PriceClass": "PriceClass_All"
				}
			}
		}
	}
}

02 CloudFormation template (YAML):

AWSTemplateFormatVersion: '2010-09-09'
	Parameters:
	KinesisStreamArn:
		Type: String
	KinesisStreamRoleArn:
		Type: String
	Resources:
	CloudFrontRealtimeLogConfig:
		Type: AWS::CloudFront::RealtimeLogConfig
		Properties:
		Name: cc-real-time-log-config
		EndPoints:
			- KinesisStreamConfig:
				RoleArn: !Ref 'KinesisStreamRoleArn'
				StreamArn: !Ref 'KinesisStreamArn'
			StreamType: Kinesis
		Fields:
			- timestamp
			- c-ip
			- time-to-first-byte
			- x-edge-location
			- x-edge-request-id
			- x-host-header
			- x-forwarded-for
			- ssl-protocol
			- ssl-cipher
		SamplingRate: 100
	CloudFrontDistribution:
		Type: AWS::CloudFront::Distribution
		Properties:
		DistributionConfig:
			Enabled: true
			Comment: CloudFront CDN Web Distribution
			Origins:
			- Id: cc-cdn-origin
				DomainName: domain.com
				CustomOriginConfig:
				HTTPPort: 80
				HTTPSPort: 443
				OriginKeepaliveTimeout: 15
				OriginSSLProtocols:
					- TLSv1.2
				OriginProtocolPolicy: https-only
			DefaultRootObject: index.html
			DefaultCacheBehavior:
			Compress: true
			AllowedMethods:
				- HEAD
				- GET
			DefaultTTL: 0
			ForwardedValues:
				Cookies:
				Forward: none
				QueryString: false
			ViewerProtocolPolicy: https-only
			RealtimeLogConfigArn: !Ref 'CloudFrontRealtimeLogConfig'
			HttpVersion: http2
			ViewerCertificate:
			CloudFrontDefaultCertificate: true
			PriceClass: PriceClass_All

Using Terraform (AWS Provider)

01 Terraform configuration file (.tf):

terraform {
	required_providers {
		aws = {
			source  = "hashicorp/aws"
			version = "~> 4.0"
		}
	}

	required_version = ">= 0.14.9"
}

provider "aws" {
	profile = "default"
	region  = "us-east-1"
}

resource "aws_cloudfront_realtime_log_config" "cloudfront-realtime-log-config" {
	name          = "cc-real-time-log-config"
	sampling_rate = 100
	fields        = ["timestamp", "c-ip", "time-to-first-byte", "x-edge-location", "x-edge-request-id", "x-host-header", "x-forwarded-for", "ssl-protocol", "ssl-cipher"]
	endpoint {
		stream_type = "Kinesis"
		kinesis_stream_config {
			role_arn   = "arn:aws:iam::123456789012:role/service-role/CloudFrontRealtimeLogConfigRole"
			stream_arn = "arn:aws:kinesis:us-east-1:123456789012:stream/cc-real-time-log-stream"
		}
	}
}

resource "aws_cloudfront_distribution" "cloudfront-distribution" {

	enabled             = true
	default_root_object = "index.html"
	comment             = "CloudFront CDN Web Distribution"

	origin {
		domain_name = "domain.com"
		origin_id   = "cc-cdn-origin"
		custom_origin_config = {
			http_port                = 80
			https_port               = 443
			origin_keepalive_timeout = 15
			origin_ssl_protocols     = ["TLSv1.2"]
			origin_protocol_policy   = "https-only"
		}
	}

	default_cache_behavior {
		compress         = true
		target_origin_id = "cc-domain-origin"
		allowed_methods  = ["GET", "HEAD"]
		cached_methods   = ["GET", "HEAD"]
		forwarded_values {
			query_string = false
			cookies {
				forward = "none"
			}
		}
		min_ttl                 = 0
		default_ttl             = 3600
		max_ttl                 = 86400
		viewer_protocol_policy  = "https-only"
		realtime_log_config_arn = aws_cloudfront_realtime_log_config.cloudfront-realtime-log-config.arn
	}

	viewer_certificate {
		cloudfront_default_certificate = true
	}

	price_class = "PriceClass_All"

}

Using AWS Console

01 Sign in to the AWS Management Console.

02 To be able to read and analyze your Amazon CloudFront real-time logs, you must send them to an Amazon Kinesis data stream. To set up a data stream, navigate to Amazon Kinesis console at https://console.aws.amazon.com/kinesis.

03 In the left navigation panel, under Amazon Kinesis, select Data streams.

04 Choose Create data stream to set up a new Amazon Kinesis data stream.

05 On the Create a data stream page, provide a unique name for the new stream in the Data stream name box, and enter the appropriate number of shards in the Number of open shards box to configure the data stream capacity. Choose Create data stream to create a new Amazon Kinesis data stream.

06 Navigate to Amazon CloudFront console at https://console.aws.amazon.com/cloudfront/v3/.

07 In the left navigation panel, under Telemetry, choose Logs.

08 Select the Real-time configurations tab and choose Create configuration to create a new real-time log configuration. The real-time log configuration contains information about which log fields you want to receive, the sampling rate for the log records, and the Kinesis data stream where you want to deliver the logs.

09 On the Create configuration setup page, perform the following actions:

  1. For Name, provide a unique name for the new real-time log configuration.
  2. For Sampling rate, enter the percentage of log records delivered, from 1% to 100%.
  3. Choose the fields that you want to include in the new configuration from the Fields dropdown list. By default, all fields are chosen. To remove a certain field from the configuration, click on the remove button (X icon) next to the field.
  4. For Endpoint, choose the Kinesis data stream where you want to receive real-time logs, created earlier in the Remediation process.
  5. For IAM role, choose Create new service role CloudFrontRealtimeLogConfigRole-name to enable the CloudFront service to create the required IAM role for you.
  6. For Distributions, select the ID of the Amazon CloudFront distribution that you want to monitor using the Real-Time Logging feature.
  7. For Cache behaviors, select the cache behavior(s) that you want to associate with the new real-time log configuration.
  8. Choose Create configuration to create the real-time log configuration and attach it to the specified Amazon CloudFront distribution. This will enable the Real-Time Logging feature for the selected distribution.

10 Repeat steps no. 7 – 9 to enable and configure real-time logging for each active Amazon CloudFront distribution available within your AWS cloud account.

Using AWS CLI

01 To read and analyze your Amazon CloudFront real-time logs, you must send the logs to an Amazon Kinesis data stream. Run create-stream command (OSX/Linux/UNIX) to create a new Amazon Kinesis data stream (the command does not produce an output):

aws kinesis create-stream
  --stream-name cc-real-time-log-stream
  --shard-count 3

02 Run describe-stream command (OSX/Linux/UNIX) with custom query filters to describe the Amazon Resource Name (ARN) of the Kinesis data stream created at the previous step:

aws kinesis describe-stream
  --stream-name cc-real-time-log-stream
  --query 'StreamDescription.StreamARN'

03 The command output should return the requested Amazon Resource Name (ARN):

"arn:aws:kinesis:us-east-1:123456789012:stream/cc-real-time-log-stream"

04 Run create-realtime-log-config command (OSX/Linux/UNIX) to create the real-time log configuration required to enable the Real-Time Logging feature for your Amazon CloudFront distribution:

aws cloudfront create-realtime-log-config
  --name cc-real-time-log-config
  --sampling-rate 100
  --fields "timestamp" "c-ip" "time-to-first-byte" "x-edge-location" "x-edge-request-id" "x-host-header" "x-forwarded-for" "ssl-protocol" "ssl-cipher"
  --end-points "StreamType=Kinesis,KinesisStreamConfig={RoleARN=arn:aws:iam::123456789012:role/service-role/CloudFrontRealtimeLogConfigRole,StreamARN=arn:aws:kinesis:us-east-1:123456789012:stream/cc-real-time-log-stream}"
  --query 'RealtimeLogConfig.ARN'

05 The command output should return the ARN of the new real-time log configuration:

"arn:aws:cloudfront::123456789012:realtime-log-config/cc-real-time-log-config"

06 Run get-distribution-config command (OSX/Linux/UNIX) to extract all the configuration information from the active Amazon CloudFront distribution that you want to reconfigure:

aws cloudfront get-distribution-config
  --id ABCDABCDABCDAB
  --query 'DistributionConfig'

07 The command output should return the requested configuration information:

{
	"CallerReference": "abcd1234-abcd-1234-abcd-1234abcd1234",
	"Aliases": {
		"Quantity": 0
	},
	"DefaultRootObject": "",
	"Origins": {
		"Quantity": 1,
		"Items": [
			{
				"Id": "cloudconformity.com",
				"DomainName": "cloudconformity.com",
				"OriginPath": "",
				"CustomHeaders": {
					"Quantity": 0
				},
				"CustomOriginConfig": {
					"HTTPPort": 80,
					"HTTPSPort": 443,
					"OriginProtocolPolicy": "http-only",
					"OriginSslProtocols": {
						"Quantity": 3,
						"Items": [
							"TLSv1",
							"TLSv1.1",
							"TLSv1.2"
						]
					},
					"OriginReadTimeout": 30,
					"OriginKeepaliveTimeout": 5
				},
				"ConnectionAttempts": 3,
				"ConnectionTimeout": 10,
				"OriginShield": {
					"Enabled": false
				}
			}
		]
	},
	"OriginGroups": {
		"Quantity": 0
	},
	"DefaultCacheBehavior": {
		"TargetOriginId": "cloudconformity.com",
		"TrustedSigners": {
			"Enabled": false,
			"Quantity": 0
		},
		"TrustedKeyGroups": {
			"Enabled": false,
			"Quantity": 0
		},
		"ViewerProtocolPolicy": "allow-all",
		"AllowedMethods": {
			"Quantity": 2,
			"Items": [
				"HEAD",
				"GET"
			],
			"CachedMethods": {
				"Quantity": 2,
				"Items": [
					"HEAD",
					"GET"
				]
			}
		},
		"SmoothStreaming": false,
		"Compress": true,
		"LambdaFunctionAssociations": {
			"Quantity": 0
		},
		"FunctionAssociations": {
			"Quantity": 0
		},
		"FieldLevelEncryptionId": "",
		"ForwardedValues": {
			"QueryString": false,
			"Cookies": {
				"Forward": "none"
			},
			"Headers": {
				"Quantity": 0
			},
			"QueryStringCacheKeys": {
				"Quantity": 0
			}
		},
		"MinTTL": 0,
		"DefaultTTL": 86400,
		"MaxTTL": 31536000
	},
	"CacheBehaviors": {
		"Quantity": 1,
		"Items": [
			{
				"PathPattern": "/images",
				"TargetOriginId": "cloudconformity.com",
				"TrustedSigners": {
					"Enabled": false,
					"Quantity": 0
				},
				"TrustedKeyGroups": {
					"Enabled": false,
					"Quantity": 0
				},
				"ViewerProtocolPolicy": "allow-all",
				"AllowedMethods": {
					"Quantity": 2,
					"Items": [
						"HEAD",
						"GET"
					],
					"CachedMethods": {
						"Quantity": 2,
						"Items": [
							"HEAD",
							"GET"
						]
					}
				},
				"SmoothStreaming": false,
				"Compress": true,
				"LambdaFunctionAssociations": {
					"Quantity": 0
				},
				"FunctionAssociations": {
					"Quantity": 0
				},
				"FieldLevelEncryptionId": "",
				"ForwardedValues": {
					"QueryString": false,
					"Cookies": {
						"Forward": "none"
					},
					"Headers": {
						"Quantity": 0
					},
					"QueryStringCacheKeys": {
						"Quantity": 0
					}
				},
				"MinTTL": 0,
				"DefaultTTL": 86400,
				"MaxTTL": 31536000
			}
		]
	},
	"CustomErrorResponses": {
		"Quantity": 0
	},
	"Comment": "",
	"Logging": {
		"Enabled": false,
		"IncludeCookies": false,
		"Bucket": "",
		"Prefix": ""
	},
	"PriceClass": "PriceClass_100",
	"Enabled": true,
	"ViewerCertificate": {
		"CloudFrontDefaultCertificate": true,
		"MinimumProtocolVersion": "TLSv1",
		"CertificateSource": "cloudfront"
	},
	"Restrictions": {
		"GeoRestriction": {
			"RestrictionType": "none",
			"Quantity": 0
		}
	},
	"WebACLId": "",
	"HttpVersion": "http1.1",
	"IsIPV6Enabled": false
}

08 Run get-distribution-config command (OSX/Linux/UNIX) to describe the current version of the configuration available for the selected distribution (i.e. eTag):

aws cloudfront get-distribution-config
  --id ABCDABCDABCDAB
  --query 'ETag'

09 The command output should return the requested information:

"AAAABBBBCCCCD"

10 Modify the configuration document returned at step no. 7 to attach the new real-time log configuration to each cache behavior associated with your CloudFront distribution, in order to send real-time logs to the specified Amazon Kinesis data stream. To attach a real-time log configuration to a custom cache behavior, add the"RealtimeLogConfigArn": "<real-time-log-config-arn>" attribute to the "CacheBehaviors.Items" array, where <real-time-log-config-arn> is the Amazon Resource Name (ARN) returned at step no. 5. Save the document with the modified distribution configuration to a JSON file named enable-real-time-logging.json:

{
	"CallerReference": "abcd1234-abcd-1234-abcd-1234abcd1234",
	"Aliases": {
		"Quantity": 0
	},
	"DefaultRootObject": "",
	"Origins": {
		"Quantity": 1,
		"Items": [
			{
				"Id": "cloudconformity.com",
				"DomainName": "cloudconformity.com",
				"OriginPath": "",
				"CustomHeaders": {
					"Quantity": 0
				},
				"CustomOriginConfig": {
					"HTTPPort": 80,
					"HTTPSPort": 443,
					"OriginProtocolPolicy": "http-only",
					"OriginSslProtocols": {
						"Quantity": 3,
						"Items": [
							"TLSv1",
							"TLSv1.1",
							"TLSv1.2"
						]
					},
					"OriginReadTimeout": 30,
					"OriginKeepaliveTimeout": 5
				},
				"ConnectionAttempts": 3,
				"ConnectionTimeout": 10,
				"OriginShield": {
					"Enabled": false
				}
			}
		]
	},
	"OriginGroups": {
		"Quantity": 0
	},
	"DefaultCacheBehavior": {
		"TargetOriginId": "cloudconformity.com",
		"TrustedSigners": {
			"Enabled": false,
			"Quantity": 0
		},
		"TrustedKeyGroups": {
			"Enabled": false,
			"Quantity": 0
		},
		"ViewerProtocolPolicy": "allow-all",
		"AllowedMethods": {
			"Quantity": 2,
			"Items": [
				"HEAD",
				"GET"
			],
			"CachedMethods": {
				"Quantity": 2,
				"Items": [
					"HEAD",
					"GET"
				]
			}
		},
		"SmoothStreaming": false,
		"Compress": true,
		"LambdaFunctionAssociations": {
			"Quantity": 0
		},
		"FunctionAssociations": {
			"Quantity": 0
		},
		"FieldLevelEncryptionId": "",
		"ForwardedValues": {
			"QueryString": false,
			"Cookies": {
				"Forward": "none"
			},
			"Headers": {
				"Quantity": 0
			},
			"QueryStringCacheKeys": {
				"Quantity": 0
			}
		},
		"MinTTL": 0,
		"DefaultTTL": 86400,
		"MaxTTL": 31536000
	},
	"CacheBehaviors": {
		"Quantity": 1,
		"Items": [
			{
				"PathPattern": "/images",
				"TargetOriginId": "cloudconformity.com",
				"TrustedSigners": {
					"Enabled": false,
					"Quantity": 0
				},
				"TrustedKeyGroups": {
					"Enabled": false,
					"Quantity": 0
				},
				"ViewerProtocolPolicy": "allow-all",
				"AllowedMethods": {
					"Quantity": 2,
					"Items": [
						"HEAD",
						"GET"
					],
					"CachedMethods": {
						"Quantity": 2,
						"Items": [
							"HEAD",
							"GET"
						]
					}
				},
				"SmoothStreaming": false,
				"Compress": true,
				"LambdaFunctionAssociations": {
					"Quantity": 0
				},
				"FunctionAssociations": {
					"Quantity": 0
				},
				"FieldLevelEncryptionId": "",
				"ForwardedValues": {
					"QueryString": false,
					"Cookies": {
						"Forward": "none"
					},
					"Headers": {
						"Quantity": 0
					},
					"QueryStringCacheKeys": {
						"Quantity": 0
					}
				},
				"MinTTL": 0,
				"DefaultTTL": 86400,
				"MaxTTL": 31536000,
				"RealtimeLogConfigArn": "<real-time-log-config-arn>"
			}
		]
	},
	"CustomErrorResponses": {
		"Quantity": 0
	},
	"Comment": "",
	"Logging": {
		"Enabled": false,
		"IncludeCookies": false,
		"Bucket": "",
		"Prefix": ""
	},
	"PriceClass": "PriceClass_100",
	"Enabled": true,
	"ViewerCertificate": {
		"CloudFrontDefaultCertificate": true,
		"MinimumProtocolVersion": "TLSv1",
		"CertificateSource": "cloudfront"
	},
	"Restrictions": {
		"GeoRestriction": {
			"RestrictionType": "none",
			"Quantity": 0
		}
	},
	"WebACLId": "",
	"HttpVersion": "http1.1",
	"IsIPV6Enabled": false
}

11 Run update-distribution command (OSX/Linux/UNIX) using the document modified at the previous step (i.e. enable-real-time-logging.json) as the distribution configuration document, to reconfigure the selected Amazon CloudFront distribution in order to enable the Real-Time Logging feature. --if-match parameter represents the current version of the configuration, returned at step no. 9:

aws cloudfront update-distribution
  --id ABCDABCDABCDAB
  --if-match AAAABBBBCCCCD
  --distribution-config file://enable-real-time-logging.json
  --query 'Distribution.Status'

12 The command output should return the status of the modified CloudFront distribution:

"InProgress"

13 Repeat steps no. 6 – 12 to enable and configure real-time logging for each active Amazon CloudFront distribution deployed in your AWS cloud account.

References

Publication date May 6, 2022