Redshift Reserved Node Recent Purchases

Risk Level: Low (generally tolerable level of risk)

Rule ID: RS-014

Ensure that all active Amazon Redshift Reserved Node (RN) purchases are reviewed every 7 days to make sure that no unwanted RN purchase has been placed recently.

This rule can help you with the following compliance standards:

NIST4

For further details on compliance standards supported by Conformity, see here.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Cost
optimisation

By checking your Redshift RN purchases on a regular basis you can detect and cancel any unwanted purchases placed within your AWS account in order to avoid unexpected charges on your AWS bill.

Note: You can change the default threshold value (7 days) for the review time frame within the rule settings available on the Cloud Conformity console.

Audit

To identify the active Redshift Reserved Node purchases placed within your AWS account for review purposes, perform the following:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Redshift dashboard at https://console.aws.amazon.com/redshift/.

03 In the left navigation panel, click Reserved Nodes.

04 Choose the active Redshift Reserved Node that you want to examine.

05 Click Show or Hide Item Details button to expand the details panel and check the Start Date attribute value (e.g. March 17, 2017 at 10:12:39 AM UTC+3). If the Start Date value indicates a recent Redshift RN purchase request (i.e. request placed in the last 7 days) and you are unaware of this purchase, check your AWS CloudTrail logs or contact Amazon Web Services using the Support Center console to solve the unwanted Redshift RN purchase issue (see Remediation/Resolution section for more details).

06 Repeat step no. 4 and 5 to verify the purchase request date for other Redshift Reserved Nodes available within the selected region.

07 Change the AWS region from the navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run describe-reserved-nodes command (OSX/Linux/UNIX) using custom query filters to list the IDs of all existing Redshift RN purchases, available in the selected AWS region:

aws redshift describe-reserved-nodes
    --region us-east-1
    --output table
    --query 'ReservedNodes[*].ReservedNodeId'

02 The command output should return a table with the requested Reserved Node(s) ID(s):

aws redshift describe-reserved-nodes
----------------------------------------
|           ReservedNodeIds            |
+--------------------------------------+
| be733e93-512a-171d-bc5b-31e2e30bd553 |
| f6568e2e-dc01-ad65-735d-f6a3e931eb19 |
+--------------------------------------+

03 Run again describe-reserved-nodes command (OSX/Linux/UNIX) using the ID of your AWS Redshift RN returned at the previous step as identifier and required filtering to expose the date at which the Redshift Reserved Node purchase request was placed:

aws redshift describe-reserved-nodes
    --region us-east-1
    --reserved-node-id be733e93-512a-171d-bc5b-31e2e30bd553
    --query 'ReservedNodes[*].StartTime'

04 The command output should return the date when the requested Redshift reservation started:

[
    "StartTime": "2017-03-17T010:12:39.112Z"
]

If the date returned as StartTime attribute value indicates a recent Redshift RN purchase request (i.e. request placed in the last 7 days) and you are unaware of this purchase, verify your AWS CloudTrail logs or contact AWS using the Support Center console to solve the issue.

05 Repeat step no. 3 and 4 to check the purchase request timestamp for other Redshift Reserved Nodes (RNs) available within the selected region.

06 Change the AWS region by updating the --region command parameter value and perform the entire audit process for other regions.

Remediation / Resolution

Case A: Verify AWS Cloudtrail logs (if Cloudtrail service is enabled) from the date when the Redshift RN purchase request was placed to determine the request origin and context. To find and analyze the necessary Redshift API logging data, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to CloudTrail dashboard at https://console.aws.amazon.com/cloudtrail/.

03 In the left navigation panel, select Trails.

04 Under Name column, select the trail name that you need to examine, available in the same AWS region with the identified Redshift RN purchase.

05 Within Storage location section check the name of the S3 bucket used to store the trail log data.

06 Now navigate to S3 dashboard at https://console.aws.amazon.com/s3/.

07 Select the S3 bucket used for CloudTrail logging and use the date/time bucket name format (e.g. cloudtrail-logging-bucket/AWSLogs/123456789012/CloudTrail/us-east-1/2017/03/17) to open the right log file for analysis.

08 Based on the log file name (i.e. 123456789012_CloudTrail_us-east-1_20170317T1012Z_11tQnvmb05cuBxKd6.json.gz), identify the CloudTrail log file that contains the API activity recorded on the same date as the unwanted Redshift RN purchase request, click the Actions dropdown button from the dashboard top menu and select Open to download and open the log file in your web browser.

09 Once the right CloudTrail log file is opened, search for the following attributes in order to identify the necessary log record:

"eventSource":"redshift.amazonaws.com" – for the name of the AWS service used to place the RN purchase request.
"eventName":"PurchaseReservedNodeOffering" – for the name of the AWS API action/command used to place the RN purchase request.
"eventTime":"2017-03-17T10:12:39.112Z" – for the time when the Redshift RN purchase request was placed.

10 Identify the right CloudTrail log record based on the attributes listed at the previous step and verify the "userIdentity" attribute value to determine your unwanted Redshift RN purchase request origin and context.

11 Repeat steps no. 7 – 10 to verify the request origin and context for other unwanted Redshift Reserved Node (RN) purchases placed in the selected region.

12 Change the AWS region from the navigation bar and repeat the entire process for other regions.

Using AWS CLI

01 Run describe-trails command (OSX/Linux/UNIX) to list the names of all CloudTrail trails currently available within the selected AWS region:

aws cloudtrail describe-trails
    --region us-east-1
    --output table
    --query 'trailList[*].Name'

02 The command output should return a table with the requested trail name(s):

---------------------------
|     DescribeTrails      |
+-------------------------+
|  cc-env-cloud-trail     |
+-------------------------+

03 Run again describe-trails command (OSX/Linux/UNIX) using the name of the trail returned at the previous step and custom query filters to get the name of the S3 bucket utilized to store the log files for the selected AWS CloudTrail trail:

aws cloudtrail describe-trails
    --region us-east-1
    --trail-name-list cc-env-cloud-trail
    --query 'trailList[*].S3BucketName'

04 The command output should return the name of the requested S3 bucket:

[
    "cc-env-trail-logs"
]

05 Now run list-objects command (OSX/Linux/UNIX) to list the names of all S3 objects available in the selected S3 bucket:

aws s3api list-objects
    --region us-east-1
    --bucket cc-env-trail-logs
    --query 'Contents[].Key'

06 The command output should expose the name of each S3 object (i.e. CloudTrail log file) currently available within the selected S3 bucket:

[
    "AWSLogs/123456789012/CloudTrail/us-east-1/2017/03/17/123456789012_
     CloudTrail_us-east-1_20170317T1012Z_11tQnvmb05cuBxKd6.json.gz",
    "AWSLogs/123456789012/CloudTrail/us-east-1/2017/03/17/123456789012_
     CloudTrail_us-east-1_20170317T1012Z_65ZWtvmCqOcuwXIo.json.gz",
    "AWSLogs/123456789012/CloudTrail/us-east-1/2017/03/17/123456789012_
     CloudTrail_us-east-1_20170317T1012Z_ntG0rPDvTVMKJdLe.json.gz",
    "AWSLogs/123456789012/CloudTrail/us-east-1/2017/03/17/123456789012_
     CloudTrail_us-east-1_20170317T1012Z_00euohtdKFqkNjPH.json.gz"
]

07 Run get-object command (OSX/Linux/UNIX) to get the right CloudTrail log file (e.g. 20170317T1012Z_11tQnvmb05cuBxKd6.json.gz) from the specified S3 bucket and download it to your machine:

aws s3api get-object
    --region us-east-1
    --bucket env-cloudtrail-logs
    --key AWSLogs/123456789012/CloudTrail/us-east-1/2017/03/17/123456789012_CloudTrail_us-east-1_20170317T1012Z_11tQnvmb05cuBxKd6.json.gz 20170317T1012Z_11tQnvmb05cuBxKd6.json.gz

08 The command output should return the GET request metadata:

{
    "AcceptRanges": "bytes",
    "ContentType": "application/json",
    "LastModified": "Sat, 17 Mar 2017 10:12:39 GMT",
    "ContentLength": 5980,
    "ContentEncoding": "gzip",
    "ETag": "\"fca6fa01349b8713ec69e0e4cf6971b8\"",
    "ServerSideEncryption": "AES256",
    "Metadata": {}
}

09 Now extract and open the required CloudTrail log file, downloaded at the previous step (e.g. 20170317T1012Z_11tQnvmb05cuBxKd6.json.gz), in your preferred text editor.

10 Once the log file is opened, search for the following attributes in order to identify the necessary log record:

"eventSource":"redshift.amazonaws.com" – for the name of the AWS service used to place the RN purchase request.
"eventName":"PurchaseReservedNodeOffering" – for the name of the AWS API action/command used to place the RN purchase request.
"eventTime":"2017-03-17T10:12:39.112Z" – for the time when the Redshift RN purchase request was placed.

11 Identify the right CloudTrail log record based on the attributes listed at the previous step and verify the "userIdentity" attribute value to determine your unwanted Redshift RN purchase request origin and context.

12 Repeat steps no. 7 – 11 to verify the request origin and context for other unwanted Redshift Reserved Node (RN) purchases available within the selected region.

13 Change the AWS region by updating the --region command parameter value and perform the entire remediation process for other regions.

Case B: To mitigate unwanted Redshift Reserved Node purchase requests you can contact Amazon Web Services and ask for RN purchases cancellation. To create the necessary case through the AWS Support Center, perform the following:

Note: Requesting Amazon to cancel your unwanted Redshift RN purchase requests using AWS Management Console or AWS API via Command Line Interface (CLI) is not currently supported.

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to AWS Support Center page at https://console.aws.amazon.com/support/.

03 On Support Center page, click Create case button to initiate the process.

04 On the Create Case page, perform the following:

Under Regarding, select Account and Billing Support option.
Choose Billing from the Service dropdown list to send your request to AWS Billing and Cost Management service.
Select Reserved Nodes from the Category dropdown list.
Inside the Subject box, enter a subject for your request such as "Cancel unwanted AWS Redshift Reserved Node purchase".
Within Description textbox, provide the reason why do you need to cancel your recent Redshift RN purchase and explain how and when this unwanted purchase request was placed. This will help AWS support team to evaluate properly your request.
Under Contact method, select a preferred contact method that AWS support team can use to respond to your request. If your need is urgent, choose Phone as contact method to request a direct phone call.
Click Submit to send the cancellation request for your unwanted Redshift Reserved Node purchase to Amazon Web Services. A customer support representative will contact you shortly.

References

AWS Documentation
Purchasing Amazon Redshift Reserved Nodes
Purchasing a Reserved Node Offering with the Amazon Redshift Console

AWS Command Line Interface (CLI) Documentation
redshift
describe-reserved-nodes
cloudtrail
describe-trails
s3api
list-objects
get-object

Publication date Mar 14, 2017

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS Console

Using AWS CLI

Using AWS Console

References

Related Redshift rules