Use the Conformity Knowledge Base AI to help improve your Cloud Posture

OpenSearch Reserved Instance Lease Expiration In The Next 30 Days

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)
Rule ID: ES-017

Ensure that your AWS Elasticsearch Reserved Instances (RIs) are renewed before expiration in order to get a significant discount on the hourly charges. The renewal process consists of purchasing another Elasticsearch RIs so that Amazon can keep charging you based on the chosen reservation term. The default threshold for the number of days before expiration when this conformity rule check is performed is 30 days.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Cost
optimisation

Reserved Instances (RIs) offer significant discounts compared to standard On-Demand Instances, therefore with this billing method in use you can optimize your Amazon Elasticsearch service costs based on your expected compute resource usage. Since Elasticsearch Reserved Instances are not renewed automatically, purchasing another RIs before expiration will guarantee their billing at a discounted hourly rate.

Note: You can change the default threshold for the number of days before expiration on Cloud Conformity account dashboard and set your own value (in days), based on your needs.


Audit

To determine the expiration date for your AWS Elasticsearch Reserved Instances, perform the following:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Elasticsearch (ES) dashboard at https://console.aws.amazon.com/es/.

03 In the left navigation panel, choose Reserved Instances.

04 Select the Reserved Instance (RI) that you want to examine and check the value listed within the Remaining Days column. If the number of days displayed in this column is less than 30 (the default threshold used when no custom threshold value is configured), the selected Amazon Elasticsearch Reserved Instance (RI) is about to expire, therefore the resource must be renewed to keep it running at the current discounted hourly rate (see Remediation/Resolution section for the renewal process).

05 Repeat step no. 4 to determine the expiration date for other AWS Elasticsearch RIs provisioned in the current region.

06 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using custom query filters to list the IDs of all Elasticsearch Reserved Instances available in the selected AWS region:

aws es describe-reserved-elasticsearch-instances
	--region us-east-1
	--output table
	--query 'ReservedElasticsearchInstances[*].ReservedElasticsearchInstanceId'

02 The command output should return a table with the requested RI IDs:

----------------------------------------
|   ReservedElasticsearchInstanceIds   |
+--------------------------------------+
| aaaabbbb-aaaa-bbbb-cccc-aaaabbbbcccc |
| abcdabcd-abcd-abcd-abcd-abcdabcdabcd |
+--------------------------------------+

03 Run describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using the ID of the Elasticsearch Reserved Instance that you want to examine as identifier and appropriate filtering to reveal the date when the selected RI was purchased and the reservation duration (the commitment term in seconds) in order to determine the reservation expiration date:

aws es describe-reserved-elasticsearch-instances
	--region us-east-1
	--reserved-elasticsearch-instance-id aaaabbbb-aaaa-bbbb-cccc-aaaabbbbcccc
	--query 'ReservedElasticsearchInstances[*].[StartTime,Duration]'

04 The command output should return the timestamp (date) at which the reservation started and the commitment term in seconds (31536000 for 1 year, 94608000 for 3 years):

[
    "StartTime": 1507369500.229,
    "Duration": 31536000
]

05 The value returned for "StartTime" attribute at the previous step is using the Unix/epoch time format, which represents the number of seconds that have passed since midnight UTC of 1 January 1970. To convert the returned value into a human-readable format, run the following command (replace the Unix timestamp with your own timestamp returned as value for the "StartTime" attribute):

date -d @1507369500.229

06 The command output should return the RI purchasing date in a human-readable format:

Sat Oct 07 09:45:00 UTC 2017

Based on the timestamp returned at the previous step and the duration value returned at step no. 4, determine the number of days remained until the reservation expires. If the number of days remained is less than 7, the selected AWS Elasticsearch Reserved Instance (RI) is about to expire, therefore the RI needs to be renewed in order to maintain the current discount rate.

07 Repeat steps no. 3 – 6 to determine the expiration date for other AWS Elasticsearch RIs provisioned in the current region.

08 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 7 to perform the entire audit process for other regions.

Remediation / Resolution

To renew your Elasticsearch Reserved Instances before their reservation expire, you need to repurchase them using the same offering configuration (AWS region, instance class, term, etc). To renew your existing AWS Elasticsearch RIs in order to avoid On-Demand rate charges when the current reservation expires, perform the following:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Elasticsearch (ES) dashboard at https://console.aws.amazon.com/es/.

03 In the left navigation panel, choose Reserved Instances.

04 Select the Elasticsearch Reserved Instance that you want to renew and note its configuration attributes (i.e. instance class, instance size, term and so on).

05 Click Purchase reserved instances button from the dashboard top menu to initiate the reservation process.

06 On Purchase reserved instances page, provide the following RI offering details:

  1. Specify a unique name, required to identify the reservation, in the Reservation name box.
  2. From Instance class dropdown list, select the same instance class used by the Elasticsearch RI that you want to renew.
  3. From Instance size dropdown list, select the same instance size as the one identified at step no. 4.
  4. Within the Number of instances box, enter the appropriate number of instances, identified at step no. 4.
  5. From Term dropdown list, select the length of time for the new reservation based on your requirements.
  6. From Payment option dropdown list, choose the payment type that you want to use for this reservation (e.g. Partial Upfront).
  7. Click Submit to continue the process.

07 On Elasticsearch RI purchase summary page, carefully review the instance(s) configuration information then click Purchase to repurchase your AWS Elasticsearch Reserved Instance.

08 Repeat steps no. 4 – 7 to renew (repurchase) other Amazon Elasticsearch RIs that are about to expire, available in the current region.

09 Change the AWS region from the navigation bar and repeat the process for other regions.

Using AWS CLI

01 Execute describe-reserved-elasticsearch-instances command (OSX/Linux/UNIX) using the ID of the Elasticsearch RI that you want to renew as identifier (see Audit section part II to identify the right resource) and custom query filters to get the existing instance offering ID. The ID will be used later as filter parameter to find the offering(s) that match the existing RI configuration attributes:

aws es describe-reserved-elasticsearch-instances
	--region us-east-1
	--reserved-elasticsearch-instance-id aaaabbbb-aaaa-bbbb-cccc-aaaabbbbcccc
	--query 'ReservedElasticsearchInstances[*].ReservedElasticsearchInstanceOfferingId'

02 The command output should return the selected Reserved Instance offering ID:

[
    "abcd12345-1234-abcd-1234-abcd1234abca"
]

03 Before you repurchase your AWS Elasticsearch Reserved Instance, run describe-reserved-elasticsearch-instance-offerings command (OSX/Linux/UNIX) to obtain the pricing details and information about available Elasticsearch RI offerings. Use the ID returned at the previous step as identifier to match the specifications of the RI that you want to repurchase:

aws es describe-reserved-elasticsearch-instance-offerings
	--region us-east-1
	--reserved-elasticsearch-instance-offering-id abcd12345-1234-abcd-1234-abcd1234abca

04 The command output should return the requested RI offering(s) information:

{
  "ReservedElasticsearchInstanceOfferings": [
    {
      "FixedPrice": 443.00,
      "ReservedElasticsearchInstanceOfferingId": "abcd12345-1234-abcd-1234-abcd1234abca",
      "RecurringCharges": [
        {
          "RecurringChargeAmount": 0.051,
          "RecurringChargeFrequency": "Hourly"
        }
      ],
      "UsagePrice": 0.0,
      "PaymentOption": "PARTIAL_UPFRONT",
      "Duration": 31536000,
      "ElasticsearchInstanceType": "m4.large.elasticsearch",
      "CurrencyCode": "USD"
    }
  ]
}

05 Run purchase-reserved-elasticsearch-instance-offering command (OSX/Linux/UNIX) to purchase the Elasticsearch Reserved Instance(s) based on the offering information returned at the previous step. In the following example, the purchase-reserved-elasticsearch-instance-offering command request to purchase 3 Elasticsearch RIs using an offering identified by the ID abcd12345-1234-abcd-1234-abcd1234abca:

aws es purchase-reserved-elasticsearch-instance-offering
	--region us-east-1
	--reserved-elasticsearch-instance-offering-id abcd12345-1234-abcd-1234-abcd1234abca
	--reservation-name cc-project5-es-reservation
	--instance-count 3

06 The command output should return the reservation request metadata:

{
  "ReservationName": "cc-project5-es-reservation",
  "ReservedElasticsearchInstanceId": "aabbccdd-aabb-ccdd-eeff-aabbccddeeff"
}

07 Repeat steps no. 1 – 6 to renew (repurchase) other Amazon Elasticsearch RIs that are about to expire, available in the selected region.

08 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 – 7 to perform the entire process for other regions.

References

Publication date Oct 29, 2018