Review Reserved Instance Purchases

Risk Level: Low (generally tolerable level of risk)

Rule ID: ES-020

Ensure that all the active Amazon OpenSearch Reserved Instance (RI) purchases are reviewed every 7 days to make sure that no unwanted RI purchase has been placed recently.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Cost
optimisation

By checking your Amazon OpenSearch Reserved Instance purchases on a regular basis you can detect and cancel any unwanted purchases placed within your AWS cloud account and avoid unexpected charges on your AWS bill.

Note: You can change the default threshold value (7 days) for the review time frame within the conformity rule settings, using the Trend Cloud One™ – Conformity console.

Audit

To identify the OpenSearch Reserved Instance purchases placed recently within your AWS cloud account for review purposes, perform the following operations:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon OpenSearch console at https://console.aws.amazon.com/esv3/.

03 In the main navigation panel, under Domains, select Reserved Instance leases.

04 Choose the Reserved Instance (RI) that you want to examine and check the value available in the Start date column to determine when the RI was purchased. If the Start date value indicates a recent purchase request (i.e. a request placed in the past 7 days) and you are unaware of this purchase, check your Amazon CloudTrail trail logs or contact AWS Support Center to solve the purchase issue for the selected, unwanted Amazon OpenSearch reservation.

05 Repeat step no. 4 for each Amazon OpenSearch Reserved Instance available within the current AWS region.

06 Change the AWS cloud region from the console navigation bar and repeat the Audit process for other regions.

Using AWS CLI

01 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) with custom query filters to list the identifier (ID) of each Amazon OpenSearch Reserved Instance available in the selected AWS region:

aws es describe-reserved-elasticsearch-instances
  --region us-east-1
  --output table
  --query 'ReservedOpenSearchInstances[*].ReservedOpenSearchInstanceId'

02 The command output should return a table with the requested Reserved Instance IDs:

----------------------------------------
|    ReservedOpenSearchInstanceIds     |
+--------------------------------------+
| aabbccdd-aabb-ccdd-1234-aabbccdd1234 |
| abcdabcd-abcd-abcd-abcd-abcdabcdabcd |
+--------------------------------------+

03 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using the ID of the OpenSearch Reserved Instance (RI) that you want to examine as the identifier parameter and custom query filters to obtain the date when the selected RI was purchased:

aws es describe-reserved-elasticsearch-instances
  --region us-east-1
  --reserved-elasticsearch-instance-id aabbccdd-aabb-ccdd-1234-aabbccdd1234
  --query 'ReservedOpenSearchInstances[*].StartTime'

04 The command output should return the timestamp (date) at which the reservation started:

[
	"StartTime": 1539274423.430
]

05 The value returned for "StartTime" attribute at the previous step is using the Unix time format, which represents the number of seconds that have passed since midnight UTC of 1 January 1970. To convert the returned value into a human-readable format, run the following command (replace the Unix timestamp with your own timestamp returned as value for the "StartTime" attribute):

date -d @1507369500.430

06 The command output should return the purchasing date in a human-readable format:

Fri Dec 17 10:11:30 UTC 2021

If the date/time returned by the date command output at the previous step indicates a recent purchase request (i.e. a request placed in the past 7 days) and you are unaware of this purchase, check your Amazon CloudTrail trail logs or contact AWS Support Center to solve the purchase issue for the selected, unwanted Amazon OpenSearch reservation.

07 Repeat steps no. 3 – 6 for each Amazon OpenSearch Reserved Instance available in the selected AWS region.

08 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 7 to perform the Audit process for other AWS regions.

Remediation / Resolution

Case A: Check the Amazon CloudTrail trail logs from the date when the OpenSearch Reserved Instance purchase request was placed to determine the request context and origin. To find and analyze the required OpenSearch API logging data, perform the following actions:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon CloudTrail console at https://console.aws.amazon.com/cloudtrail/.

03 In the navigation panel, under CloudTrail, choose Trails.

04 Click on the name of the Amazon CloudTrail trail that you want to examine, available in the same AWS region with the Reserved Instance purchase that you want to analyze.

05 In the General details section, choose Edit and copy the name of the associated S3 bucket available in the Trail log bucket name box.

06 Navigate to Amazon S3 console at https://console.aws.amazon.com/s3/.

07 Paste the name of bucket copied at step no. 5 in the Find buckets by name box and click on the name of the returned S3 bucket to access the bucket objects (log files).

08 Use the following format to access the right folder for analysis: <cloudtrail-logging-bucket>/AWSLogs/<aws-account-id>/CloudTrail/<aws-region> /<year>/<month>/<day>.

09 Based on the log file name identify the CloudTrail log file that contains the API activity recorded on the same date as the unwanted RI purchase request, select the log file, and choose Download to download the required log file.

10 Open the log file in your web browser and search for the following attributes to identify the necessary log record:

"eventSource":"es.amazonaws.com" – for the name of the AWS service used to place the Redshift Instance (RI) purchase request.
"eventName":"PurchaseReservedOpenSearchInstance" – for the name of the AWS API action/command used to place the RI purchase request.
"eventTime":"2021-12-17T10:11:30.112Z" – for the time when the RI purchase request was placed.

11 Identify the right CloudTrail log record based on the attributes listed at the previous step and check the "userIdentity" attribute value to determine the origin and context for the unwanted OpenSearch Reserved Instance purchase request.

12 Repeat steps no. 8 – 11 for each unwanted OpenSearch Reserved Instance purchase request placed in the selected AWS region.

13 Change the AWS cloud region from the console navigation bar and repeat the Remediation process for other regions.

Using AWS CLI

01 Run list-trails command (OSX/Linux/UNIX) with custom query filters to list the names of all the Amazon CloudTrail trails created for your AWS cloud account:

aws cloudtrail list-trails
  --region us-east-1
  --query 'Trails[*].Name'

02 The command output should return an array with the requested CloudTrail trail names:

[
	"cc-main-cloud-trail",
	"cc-data-events-trail"
]

03 Run describe-trails command (OSX/Linux/UNIX) using the name of the Amazon CloudTrail trail that you want to examine as the identifier parameter and custom query filters to describe the name of the S3 bucket configured to store logs for the selected trail:

aws cloudtrail describe-trails
  --region us-east-1
  --trail-name-list cc-main-cloud-trail
  --query 'trailList[*].S3BucketName'

04 The command output should return the name of the associated bucket:

[
	"cc-main-cloudtrail-logs"
]

05 Run list-objects command (OSX/Linux/UNIX) to list the name of each S3 object available in the selected Amazon S3 bucket:

aws s3api list-objects
  --region us-east-1
  --bucket cc-main-cloud-trail
  --query 'Contents[].Key'

06 The command output should describe the name of each CloudTrail log file available in the selected S3 bucket:

[
	"AWSLogs/123456789012/CloudTrail/us-east-1/2021/12/17/123456789012_CloudTrail_us-east-1_202111171T1613Z_aaaabbbbccccdddd.json.gz",
	"AWSLogs/123456789012/CloudTrail/us-east-1/2021/12/17/123456789012_CloudTrail_us-east-1_20211217T1235Z_abcdabcdabcdabcd.json.gz"
]

07 Run get-object command (OSX/Linux/UNIX) to obtain and download the right CloudTrail log file from the target S3 bucket:

aws s3api get-object
  --region us-east-1
  --bucket cc-main-cloud-trail
  --key AWSLogs/123456789012/CloudTrail/us-east-1/2021/12/17/123456789012_CloudTrail_us-east-1_202111171T1613Z_aaaabbbbccccdddd.json.gz

08 The command output should return the GET request metadata, e.g.:

{
	"AcceptRanges": "bytes",
	"ContentType": "application/json",
	"LastModified": "Fri, 17 Dec 2021 11:30:39 GMT",
	"ContentLength": 5980,
	"ContentEncoding": "gzip",
	"ETag": "\"abcd1234abcd1234abcd1234abcd1234\"",
	"ServerSideEncryption": "AES256",
	"Metadata": {}
}

09 Open the log file downloaded at the previous steps in your preferred text editor and search for the following attributes in order to identify the necessary log record:

"eventSource":"es.amazonaws.com" – for the name of the AWS service used to place the Redshift Instance (RI) purchase request.
"eventName":"PurchaseReservedOpenSearchInstance" – for the name of the AWS API action/command used to place the RI purchase request.
"eventTime":"2021-12-17T10:11:30.112Z" – for the time when the RI purchase request was placed.

10 Identify the right CloudTrail log record based on the attributes listed at the previous step and check the "userIdentity" attribute value to determine the origin and context for the unwanted OpenSearch Reserved Instance purchase request.

11 Repeat steps no. 7 – 10 for each unwanted OpenSearch Reserved Instance purchase request placed in the selected AWS region.

12 Change the AWS cloud region by updating the --region command parameter value and perform the Remediation process for other regions.

Case B: To mitigate unwanted Amazon OpenSearch Reserved Instance purchase requests you can contact AWS Support and ask for Reserved Instance purchase cancellation. To create the necessary case using the AWS Support Center, perform the following actions:

Note: Creating a support case to request Reserved Instance purchase cancellation using the AWS Command Line Interface (AWS CLI) is not currently supported.

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to AWS Support Center console at https://console.aws.amazon.com/support/.

03 In the Open support cases section, choose Create case to initiate the request process.

04 On the Create case page, perform the following actions:

Select Account and billing support option.
Select Account from the Type dropdown list.
Select Other Account Issues from the Category dropdown list.
Provide the request subject in the Subject box, e.g. "Cancel unwanted Amazon OpenSearch Reserved Instance purchase(s)".
For Description, provide a concise description where you can specify the reason why you want to cancel the recent OpenSearch Reserved Instance purchase and explain how and when this unwanted purchase request was placed. This will help the AWS support team to evaluate faster your request.
For Contact options, choose your preferred correspondence language from the Preferred contact language dropdown list, then select a preferred contact method that AWS support team can use to respond to your request from the Contact methods section.
Choose Submit to send your request to Amazon Web Services. A customer support representative should contact you shortly.

References

AWS Documentation
Amazon OpenSearch Service FAQs
Reserved Instances in Amazon OpenSearch Service
Amazon OpenSearch Service

AWS Command Line Interface (CLI) Documentation
es
describe-reserved-elasticsearch-instances
cloudtrail
describe-trails
s3api
list-objects
get-object

Publication date Oct 29, 2018

Audit

Using AWS Console

Using AWS CLI

Remediation / Resolution

Using AWS Console

Using AWS CLI

Using AWS Console

References

Related Elasticsearch rules