Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Idle AWS ElastiCache Nodes

Trend Micro Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 750 automated best practice checks.

Risk Level: High (not acceptable risk)
Rule ID: EC-015

Identify any Amazon ElastiCache cluster nodes that appear to be idle and remove them from your account to help lower the cost of your monthly AWS bill. By default, an AWS ElastiCache node is considered 'idle' when meets the following criteria:

The average CPU Utilization has been less than 2% for the last 7 days.

- The AWS CloudWatch metrics used to detect idle ElastiCache clusters are:

CPUUtilization (host-level metric) - the percentage of CPU resources used by ElastiCache cache nodes (Units: Percentage).


Note 1: You can easily change the default threshold for this rule on Cloud Conformity console and set your own value for the CPU usage in order to configure the ElastiCache nodes idleness based on your requirements.
Note 2: For this rule Cloud Conformity assumes that your AWS ElastiCache resources are tagged with 'Role' and 'Owner' tags which provide visibility into their usage profile and help you decide whether it`s safe or not to terminate them.

This rule resolution is part of the Conformity solution.

Sustainability
Cost
optimisation

Idle Amazon ElastiCache cache cluster nodes represent a good candidate to reduce your monthly AWS costs and avoid accumulating unnecessary usage charges.


Audit

Case A: To determine if you have any idle ElastiCache Memcached cluster nodes within your AWS account, perform the following:

Using AWS Console

01 Login to the AWS Management Console.

02 Navigate to ElastiCache dashboard at https://console.aws.amazon.com/elasticache/.

03 In the left navigation panel, under ElastiCache Dashboard, click Memcached to access the clusters created with Memcached in-memory cache engine.

04 Choose the active Memcached cluster that you want to examine and click on its name, e.g.

Memcached

to access the configuration page.

05 Choose Nodes tab from the dashboard top panel and select the node(s) provisioned within the selected Memcached cluster.

06 On the bottom panel, within the monitoring metrics section, click on the CPU Utilization (Percent) usage graph thumbnail to open the cache node(s) CPU usage details box. Inside the CPU Utilization (Percent) dialog box, set the following parameters:

  1. From the Statistic dropdown list, select Average.
  2. From the Time Range list, select Last 1 week.
  3. From the Period dropdown list, select 1 hour.

Once the monitoring data is loaded, verify the cluster CPU usage for the last 7 days. If the average usage (percent) has been less than 2%, e.g.

If the average usage (percent) has been less than 2%

the selected Memcached cluster node(s) qualify as candidate(s) for idle ElastiCache cluster node(s). Click X (close) to return to the ElastiCache dashboard.

07 Now determine the selected ElastiCache cluster role within the stack and its owner by checking the Role and Owner tags values applied to the cluster in order to decide whether it's safe or not to delete the resource. To check for the required tags, perform the following:

  1. Select Description tab from the dashboard top panel to access the cluster details.
  2. On the Description tab, within the Tags section: Tags, verify the available tags and their values:
    • Check the Role tag value or any Role-like tag value that can provide information about the usage profile of the ElastiCache cluster in order to decide if the resource can be terminated or not.
    • Check the Owner tag value or any Owner-like tag value that can provide the contact information (name, email, phone number) of the resource owner in order to get the confirmation to delete or not the selected ElastiCache cluster.

If the conditions outlined at step no. 6 and 7 (a and b) are met, the selected AWS ElastiCache Memcached cluster nodes are considered "idle" and the cache cluster can be terminated in order to stop incurring charges for it.

08 Repeat steps no. 4 – 7 to verify the CPU usage, the role and the owner tags for the rest of the ElastiCache Memcached clusters provisioned in the current region.

09 Change the AWS region from the navigation bar and repeat the audit process for the other regions.

Using AWS CLI

01 Run describe-cache-clusters command (OSX/Linux/UNIX) using custom query filters based on JMESPath query language (integrated by AWS CLI), to list the names of all ElastiCache Memcached clusters available within the selected region:

aws elasticache describe-cache-clusters
	--region us-east-1
	--output table
	--query 'CacheClusters[?(Engine==`memcached`)].CacheClusterId | []'

02 The command output should return a table with the requested Memcached cluster identifiers:

-----------------------
|DescribeCacheClusters|
+---------------------+
|  cc-memcached       |
|  cc-webcache-test   |
+---------------------+

03 Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the CPUUtilization metric representing the CPU usage of the selected ElastiCache Memcached cluster. The following command example returns the average CPU utilization of a cluster identified by the name "cc-memcached", usage data captured during a 7 days period, using 1 hour period as the granularity of the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name CPUUtilization
	--start-time 2017-04-18T02:15:32
	--end-time 2017-04-25T02:15:32
	--period 3600
	--namespace AWS/ElastiCache
	--statistics Average
	--dimensions Name=CacheClusterId,Value=cc-memcached

04 The command output should return the CPU usage details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-04-18T02:15:32Z",
            "Average": 1.0380,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-18T03:15:32Z",
            "Average": 1.2113,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-18T04:15:32Z",
            "Average": 1.1460,
            "Unit": "Percent"
        },

        ...

        {
            "Timestamp": "2017-04-25T00:15:32Z",
            "Average": 0.530999999999999993,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-25T01:15:32Z",
            "Average": 0.22833333333333333,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-25T02:15:32Z",
            "Average": 0.12783333333333333,
            "Unit": "Percent"
        }
    ],
    "Label": "CPUUtilization"
}

05 Run list-tags-for-resource command (OSX/Linux/UNIX) to describe the tags applied to the selected Memcached cluster. These tags are used to determine the cache cluster role within your application stack in order to decide whether it's safe or not to terminate the resource:

aws elasticache list-tags-for-resource
	--region us-east-1
	--resource-name arn:aws:elasticache:us-east-1:0123456789012:cluster:cc-memcached

06 The command output should return the requested cluster tags:

{
    "TagList": [
        {
            "Value": "web app cache layer",
            "Key": "Role"
        },
        {
            "Value": "staging",
            "Key": "Environment"
        }
    ]
}


If the data returned at step no. 4 and 6 satisfy the conditions set by the conformity rule, the selected ElastiCache Memcached cluster is considered "idle" and can be safely removed in order to reduce your AWS monthly costs.

07 Repeat steps no. 3 – 6 to verify the CPU usage and the role/owner tags for the rest of the Memcached clusters provisioned in the current region.

08 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 7 to perform the entire audit process for other regions.

Case B: To determine if you have any idle ElastiCache Redis cluster nodes within your AWS account, perform the following:

Using AWS Console

01 Login to the AWS Management Console.

02 Navigate to ElastiCache dashboard at https://console.aws.amazon.com/elasticache/.

03 In the left navigation panel, under ElastiCache Dashboard, click Redis to access the clusters created with the Redis cache engine.

04 Choose the active Redis cluster that you want to examine and click on its name, e.g.

Choose the active Redis cluster that you want to examine and click on its name

to access the configuration page.

05 Since Redis is single-threaded, the threshold is calculated as (2% / number of vCPUs). For example, if your Redis cluster is using a cache.m4.xlarge which has 4 vCPU, the CPU utilization threshold should be less than 0.5% (2% / 4 = 0.5%). Choose Description tab from the dashboard top panel and check the cluster node type listed as value for the Node type attribute. The node type value available on the Description tab will help identify the number of vCPUs used by the selected cache node. The details regarding the number of vCPUs available for each Elasticache node type can be found at this URL.

06 Choose Nodes tab from the dashboard top panel and select a node provisioned within the selected Redis cluster.

07 On the bottom panel, within the monitoring metrics section, click on the CPU Utilization (Percent) usage graph thumbnail to open the cache nodes CPU usage details box. Inside the CPU Utilization (Percent) dialog box, set the following parameters:

  1. From the Statistic dropdown list, select Average.
  2. From the Time Range list, select Last 1 week.
  3. From the Period dropdown list, select 1 hour.

Once the monitoring data is loaded, verify the cluster CPU usage for the last 7 days. If the average usage (percent) has been less than 2% for all available vCPUs, e.g. CPU usage for a cache node with 4 vCPUs:

CPU usage for a cache node with 4 vCPUs

the selected Redis node qualify as candidate for idle ElastiCache node. Click X (close) to return to the ElastiCache dashboard.

08 Now determine the selected ElastiCache node role within the stack and its owner by checking the Role and Owner tags values applied to the Redis node in order to decide whether it's safe or not to delete the resource. To check for the required tags, perform the following:

  1. Select Nodes tab from the dashboard top panel and select the node that you want to examine.
  2. Click the Manage Tags from the dashboard top menu then verify the available tags and their values:
    • Check the Role tag value or any Role-like tag value that can provide information about the usage profile of the Redis node in order to decide if the resource can be terminated or not.
    • Check the Owner tag value or any Owner-like tag value that can provide the contact information (name, email, phone number) of the resource owner in order to get the confirmation to delete or not the selected Redis cache node.

If all conditions outlined at step no. 7 and 8 are met, the selected AWS ElastiCache Redis cluster node is considered "idle" and can be terminated in order to stop incurring charges.

09 Repeat steps no. 6 – 8 to verify the CPU usage, the role and the owner tags for the rest of the Redis cache nodes provisioned within the selected cluster.

10 Repeat steps no. 4 – 9 to examine other ElastiCache Redis clusters provisioned in the current region.

11 Change the AWS region from the navigation bar and repeat the audit process for the other regions.

Using AWS CLI

01 Run describe-cache-clusters command (OSX/Linux/UNIX) using custom query filters based on JMESPath query language (integrated by AWS CLI), to list the names of all ElastiCache Redis clusters provisioned in the selected region:

aws elasticache describe-cache-clusters
	--region us-east-1
	--output table
	--query 'CacheClusters[?(Engine==`redis`)].CacheClusterId | []'

02 The command output should return a table with the requested Redis cluster identifiers:

-----------------------
|DescribeCacheClusters|
+---------------------+
|  cc-rediscache-001  |
|  cc-rediscache-002  |
+---------------------+

03 Since Redis is single-threaded, the threshold is calculated as (2% / number of vCPUs). For example, if your Redis cluster is using a cache.m4.xlarge node which has 4 vCPU, the CPU utilization threshold should be less than 0.5% (2% / 4 = 0.5%). The details regarding the number of vCPUs available for each Elasticache node type can be found at this URL. To expose the selected Redis cluster node type run describe-cache-clusters command (OSX/Linux/UNIX) using the name of the cluster returned at the previous step as identifier:

aws elasticache describe-cache-clusters
	--region us-east-1
	--cache-cluster-id cc-rediscache-001
	--query 'CacheClusters[*].CacheNodeType'

04 The command output should return node type used by the selected cache cluster:

[
    "cache.m4.xlarge"
]

05 Now run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the CPUUtilization metric representing the CPU usage of the selected Redis cluster. The following command example returns the average CPU utilization of an ElastiCache Redis cluster identified by the name "cc-rediscache-001", usage data captured during a 7 days period, using 1 hour period as the granularity of the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name CPUUtilization
	--start-time 2017-04-18T02:50:10
	--end-time 2017-04-25T02:50:10
	--period 3600
	--namespace AWS/ElastiCache
	--statistics Average
	--dimensions Name=CacheClusterId,Value=cc-redis

06 The command output should return the CPU usage details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-04-18T02:50:10Z",
            "Average": 0.33473333333333333,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-18T03:50:10Z",
            "Average": 0.21139999999999999,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-18T04:50:10Z",
            "Average": 0.32493333333333333,
            "Unit": "Percent"
        },

        ...

        {
            "Timestamp": "2017-04-25T00:50:10Z",
            "Average": 0.11999999999999993,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-25T01:50:10Z",
            "Average": 0.14833333333333333,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-04-25T02:50:10Z",
            "Average": 0.22683333333333333,
            "Unit": "Percent"
        }
    ],
    "Label": "CPUUtilization"
}

If the average usage (percent) has been less than 2% for all available vCPUs (4 vCPUs in this case), the selected Redis node qualify as candidate for idle ElastiCache node.

07 Run list-tags-for-resource command (OSX/Linux/UNIX) to describe the tags applied to the selected Redis cluster. These tags are used to determine the cache cluster role within your application stack in order to decide whether it's safe or not to terminate the resource:

aws elasticache list-tags-for-resource
	--region us-east-1
	--resource-name arn:aws:elasticache:us-east-1:0123456789012:cluster:cc-rediscache-001

08 The command output should return the requested cluster tags:

{
    "TagList": [
        {
            "Value": "Redis cache layer",
            "Key": "Role"
        },
        {
            "Value": "testing",
            "Key": "Environment"
        }
    ]
}

If the data returned at step no. 6 and 8 satisfy the conditions set by the conformity rule, the selected ElastiCache Redis cluster is considered "idle" and can be safely removed in order to reduce your AWS monthly costs.

09 Repeat steps no. 3 – 8 to verify the CPU usage and the role/owner tags for the rest of the Redis cache clusters provisioned in the current region.

10 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 9 to perform the entire audit process for other regions.

Remediation / Resolution

Option 1: Delete any AWS ElastiCache clusters that are currently running in idle mode. To remove the idle cache clusters, perform the following actions:

Using AWS Console

01 Login to the AWS Management Console.

02 Navigate to ElastiCache dashboard at https://console.aws.amazon.com/elasticache/.

03 In the left navigation panel, under ElastiCache Dashboard, perform one the following:

  1. To remove an ElastiCache Memcached cluster, choose Memcached then follow the instructions below:
    • Select the Memcached cache cluster that you want to remove (see Audit section part I to identify the right resource).
    • Click the Delete button from the dashboard top menu to initiate the removal process.
    • In the Delete Cluster dialog box, click Delete to confirm the action.
  2. To remove an ElastiCache Redis cluster, choose Redis and follow the instructions below:
    • Select the Redis cache cluster that you want to remove.
    • Click the Delete button from the dashboard top menu to initiate the removal process.
    • Inside the Delete Cluster dialog box, choose whether to create or not a final backup for the cluster by using the Create final backup dropdown menu then click the Delete button to confirm the action.

04 Repeat step no. 3 to terminate any other idle AWS ElastiCache clusters available within the current region.

05 Change the AWS region from the navigation bar and repeat the remediation process for other regions.

Using AWS CLI

01 Based on the ElastiCache in-memory engine used, perform one of following commands:

  1. To terminate an Amazon ElastiCache Memcached cluster run delete-cache-cluster command (OSX/Linux/UNIX) using the name of the cache cluster as identifier. The following command example deletes a Memcached cache cluster identified by the name "cc-memcached", available within the US East (N. Virginia) region:
    aws elasticache delete-cache-cluster
    	--region us-east-1
    	--cache-cluster-id cc-memcached
    
    • The command output should return the deleted Memcached cluster metadata:
      {
          "CacheCluster": {
              "Engine": "memcached",
              "CacheParameterGroup": {
                  "CacheNodeIdsToReboot": [],
                  "CacheParameterGroupName": "default.memcached1.4",
                  "ParameterApplyStatus": "in-sync"
              },
              "CacheClusterId": "cc-memcached",
              "PreferredAvailabilityZone": "us-east-1d",
              "CacheClusterCreateTime": "2017-01-20T10:23:21.697Z",
      
              ...
      
              "AutoMinorVersionUpgrade": true,
              "CacheClusterStatus": "deleting",
              "NumCacheNodes": 2,
              "CacheSubnetGroupName": "default",
              "EngineVersion": "1.4.34",
              "PendingModifiedValues": {},
              "PreferredMaintenanceWindow": "fri:09:00-fri:10:00",
              "CacheNodeType": "cache.m4.xlarge"
          }
      }
      
  2. To terminate an AWS ElastiCache Redis cluster (replication group) run delete-replication-group command (OSX/Linux/UNIX) using the name of the replication group as identifier. The following command example deletes a Redis cache replication group identified by the name "cc-rediscache", available within the US East (N. Virginia) region:
    aws elasticache delete-replication-group
    	--region us-east-1
    	--replication-group-id cc-rediscache
    
    • The command output should return the delete request metadata for your Redis cluster:
      {
          "ReplicationGroup": {
              "Status": "deleting",
              "AutomaticFailover": "disabled",
              "Description": " ",
              "ReplicationGroupId": "cc-rediscache",
              "PendingModifiedValues": {}
          }
      }
      

02 Repeat step no. 1 to delete any other idle AWS ElastiCache clusters provisioned within the current region.

03 Change the AWS region by updating the --region command parameter value and repeat the entire process for other regions.

Option 2: Disable the rule check. If the selected idle Amazon ElastiCache cluster is needed (its role within your application stack/environment is important), you may want turn off the conformity rule check for the specified ElastiCache cluster from the Cloud Conformity console

References

Publication date May 2, 2017

Unlock the Remediation Steps


Free 30-day Trial

Automatically audit your configurations with Conformity
and gain access to our cloud security platform.

Confirmity Cloud Platform

No thanks, back to article

You are auditing:

Idle AWS ElastiCache Nodes

Risk Level: High