Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Underutilized RDS Instance

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: High (not acceptable risk)
Rule ID: RDS-019

Identify any Amazon RDS database instances that appear to be underutilized and downsize (resize) them to help lower the cost of your monthly AWS bill. By default, an RDS database instance is considered "underutilized" when meets the following criteria:

  • The average CPU utilization has been less than 60% for the last 7 days.
  • The total number of ReadIOPS and WriteIOPS recorded per day for the last 7 days has been less than 100 on average.

The AWS CloudWatch metrics utilized to detect underused RDS instances are:
  • CPUUtilization - the percentage of CPU utilization (Units: Percent).
  • ReadIOPS and WriteIOPS - the average number of disk I/O (Input/Output) operations per second (Units: Count/Second).

Note: You can change the default threshold values for this rule on the Cloud Conformity console and set your own values for CPU utilization, and the total number of ReadIOPS and WriteIOPS to configure the underuse level for your RDS instances.

This rule can help you work with the AWS Well-Architected Framework.

This rule resolution is part of the Conformity Security & Compliance tool for AWS.

Sustainability
Cost
optimisation

Downsizing underused RDS database instances represents a good strategy for optimizing your monthly AWS costs. For example, downgrading a db.m3.large RDS MySQL database instance to a db.m3.medium instance due to CPU and IOPS underuse, you can save roughly $70 per month (as of March 2017).


Audit

To identify any underutilized RDS instances provisioned within your AWS account, perform the following actions:

Using AWS Console

01 Log in to the AWS Management Console.

02 Navigate to RDS dashboard at https://console.aws.amazon.com/rds/.

03 In the left navigation panel, under RDS Dashboard section, choose Instances.

04 Select the RDS database instance that you want to examine.

05 Click on Show Monitoring button from the dashboard top menu and select Show Multi-Graph View to expand the AWS CloudWatch monitoring panel.

06 On the monitoring panel displayed for the selected instance, perform the following actions:

  1. Click on the CPU Utilization (Percent) usage graph thumbnail to open the RDS instance CPU usage details box. Inside the CPU Utilization (Percent) dialog box, set the following parameters:
    • From the Statistic dropdown list, select Average.
    • From the Time Range list, select Last 1 Week.
    • From the Period dropdown list, select 1 Hour.

    Once the monitoring data is available, verify the instance CPU usage for the last 7 days. If the average usage (percent) has been less than 60%, e.g.  If the average usage (percent) has been less than 60%, the selected database instance qualifies as candidate for the underused instance. Click X (close) to return to the RDS dashboard.
  2. Click on the Read IOPS (Count/Second) usage graph thumbnail to open the database instance disk ReadIOPS usage details box. Inside the Read IOPS (Count/Second) dialog box, set the following parameters:
    • From the Statistic dropdown list, select Sum.
    • From the Time Range list, select Last 1 Week.
    • From the Period dropdown list, select 1 Hour.

    Once the monitoring data is loaded, verify the total number of Read operations per second for the past 7 days. If the total number of ReadIOPS has been less than 100, e.g. If the total number of ReadIOPS has been less than 100, the selected RDS instance qualifies as candidate for the underutilized database instance. Click X to close the dialog box.
  3. Click on the Write Operations (Count/Second) usage graph thumbnail to open the instance WriteIOPS usage details box. Inside the Write Operations (Count/Second) dialog box, set the following parameters:
    • From the Statistic dropdown list, select Sum.
    • From the Time Range list, select Last 1 Week.
    • From the Period dropdown list, select 1 Hour.
    Once the monitoring data is available within the CloudWatch widget, verify the total number of Write operations per second for the last 7 days. If the total number of WriteIOPS has been less than 100, e.g. If the total number of WriteIOPS has been less than 100, the selected database instance qualifies as candidate for the underused instance. Click X (close) to return to your dashboard.

If all conditions outlined at step no. 6 (a, b and c) are met, the selected RDS database instance is considered "underutilized" and should be downsized in order to stop accumulating unnecessary usage charges.

07 Repeat steps no. 4 – 6 to verify the CPU, ReadIOPS and WriteIOPS metrics usage data recorded within the selected time frame (7 days) for the rest of the RDS database instances provisioned within the current region.

08 Change the AWS region from the navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run describe-instances command (OSX/Linux/UNIX) using custom query filters to list the IDs of all the RDS database instances available in the selected region:

aws rds describe-db-instances
	--region us-east-1
	--output table
	--query 'DBInstances[*].DBInstanceIdentifier'

02 The command output should return a table with the requested IDs:

-------------------------
|  DescribeDBInstances  |
+-----------------------+
|  cc-mysql-staging-db  |
|  cc-mysql-webdev-db   |
|  cc-mysql-production  |
+-----------------------+

03 Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the CPUUtilization metric representing the CPU usage of the selected RDS instance. The following command example returns the average CPU utilization for an RDS database instance identified by the ID cc-mysql-staging-db, usage data captured during a 7-day time frame, using 1 hour time range as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name CPUUtilization
	--start-time 2017-03-05T11:18:00
	--end-time 2017-03-12T11:18:00
	--period 3600
	--namespace AWS/RDS
	--statistics Average
	--dimensions Name=DBInstanceIdentifier,Value=cc-mysql-staging-db

04 The command output should return the CPU usage details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-03-05T11:18:00Z",
            "Average": 1.2085,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-03-05T12:18:00Z",
            "Average": 1.033499999999999995,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-03-05T13:18:00Z",
            "Average": 1.10425,
            "Unit": "Percent"
        },

        ...

        {
            "Timestamp": "2017-03-12T09:18:00Z",
            "Average": 2.430999999999999993,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-03-12T10:18:00Z",
            "Average": 2.92833333333333333,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-03-12T11:18:00Z",
            "Average": 1.52783333333333333,
            "Unit": "Percent"
        }
    ],
    "Label": "CPUUtilization"
}

If the average CPU usage data returned is less than 60%, the selected RDS instance qualifies as candidate for the underused database instance.

05 Run again get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the ReadIOPS metric, representing the number of Read I/O operations per second. The following command example returns the total number of ReadIOPS used by an AWS RDS instance identified by the ID cc-mysql-staging-db, usage data captured during a 7-day time period, using a time interval of 1 hour as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name ReadIOPS
	--start-time 2017-03-05T12:32:05
	--end-time 2017-03-12T12:32:05
	--period 3600
	--namespace AWS/RDS
	--statistics Sum
	--dimensions Name=DBInstanceIdentifier,Value=cc-mysql-staging-db

06 The command output should return the ReadIOPS usage details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-03-05T12:32:05Z",
            "Sum": 10.23400416762398904,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-05T13:32:05Z",
            "Sum": 11.4499758337361044,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-05T14:32:05Z",
            "Sum": 12.5666,
            "Unit": "Count/Second"
        },

        ...

        {
            "Timestamp": "2017-03-12T10:32:05Z",
            "Sum": 13.0999,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-12T11:32:055Z",
            "Sum": 13.4499758337361044,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-12T12:32:05Z",
            "Sum": 12.2000366678889297,
            "Unit": "Count/Second"
        }
    ],
    "Label": "ReadIOPS"
}

If the total number of ReadIOPS has been less than 100 in the last 7 days, the selected RDS database instance qualifies as candidate for the underutilized instance.

07 Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the WriteIOPS metric, representing the number of Write I/O operations per second. The following command example returns the total number of WriteIOPS used by an AWS RDS instance identified by the ID cc-mysql-staging-db, IOPS usage data captured during a 7-day time frame, using 1 hour time range as the granularity for the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name WriteIOPS
	--start-time 2017-03-05T14:10:56
	--end-time 2017-03-12T14:10:56
	--period 3600
	--namespace AWS/RDS
	--statistics Sum
	--dimensions Name=DBInstanceIdentifier,Value=cc-mysql-staging-db

08 The command output should return the WriteIOPS usage details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-03-05T14:10:56Z",
            "Sum": 0.8340,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-05T15:10:56Z",
            "Sum": 0.5499,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-05T16:10:56Z",
            "Sum": 0.7666,
            "Unit": "Count/Second"
        },

        ...

        {
            "Timestamp": "2017-03-12T12:10:56Z",
            "Sum": 0.8534279999999999,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-12T13:10:56Z",
            "Sum": 0.84997,
            "Unit": "Count/Second"
        },
        {
            "Timestamp": "2017-03-12T14:10:56Z",
            "Sum": 0.8499758337361044,
            "Unit": "Count/Second"
        }
    ],
    "Label": "WriteIOPS"
}

If the total number of WriteIOPS has been less than 100 within the past 7 days, the selected RDS instance qualifies as candidate for the underused database instance.
If the usage data returned at steps no. 3 - 8 satisfy the conditions set by the conformity rule, the selected RDS database instance is considered "underutilized" and can be safely downsized in order to reduce the AWS RDS monthly usage costs.

09 Repeat steps no. 3 – 8 to verify the CPU, ReadIOPS and WriteIOPS metrics usage data recorded within the selected time frame for the rest of the RDS instances provisioned in the current region.

10 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 9 to perform the entire audit process for other regions.

Remediation / Resolution

Option 1: Downsize (resize) the underused RDS instances provisioned within your AWS account. To resize any RDS database instance that is currently running in "underutilized" mode, perform the following actions:

(!) Important note: the following process assumes that the RDS database instances selected for downsize are NOT currently used in production or for critical operations. To resize production instances without any downtime, you should create a full snapshot of your current instance and use it to launch a new database instance with the required type.

Using AWS Console

01 Log in to the AWS Management Console.

02 Navigate to RDS dashboard at https://console.aws.amazon.com/rds/.

03 In the navigation panel, under RDS Dashboard section, choose Instances.

04 Select the underused RDS database instance that you want to resize (see Audit section part I to identify the right resource).

05 Click the Instance Actions button from the dashboard top menu and select Modify.

06 On the Modify DB Instance: <instance identifier> page, under Instance Specifications section, select the database instance type to downsize to (e.g. db.m3.medium) from the DB Instance Class dropdown list (see RDS Instance Types page available at this URL to help you choose the right database instance type).

07 At the bottom of the page, check Apply Immediately to apply the instance type (class) change immediately.

08 Click Continue to access the database instance review page.

09 Review the type/class changes then click Modify DB Instance. During the resize process the instance status should change from available to modifying and back to available (it may take few minutes depending on the database instance configuration). Once the downsize is complete, the RDS instance type should be updated on the details panel.

10 Repeat steps no. 4 - 9 to downsize (resize) any other underutilized RDS database instances provisioned within the current region.

11 Change the AWS region from the navigation bar and repeat the remediation/resolution process for other regions.

Using AWS CLI

01 Run modify-db-instance command (OSX/Linux/UNIX) to resize the selected RDS database instance to the desired type/class. The following command example make use of --apply-immediately parameter to apply the type change asynchronously and trigger an immediate outage. To avoid any downtime, you can skip adding the --apply-immediately parameter to the command request and AWS RDS service will apply the change during the next maintenance window:

aws rds modify-db-instance
	--region us-east-1
	--db-instance-identifier cc-mysql-staging-db
	--db-instance-class db.m3.medium
	--apply-immediately

02 The command output should return the new configuration metadata for the specified RDS database:

{
    "DBInstance": {
        "PubliclyAccessible": true,
        "MasterUsername": "staging_usr",
        "MonitoringInterval": 0,
        "LicenseModel": "general-public-license",
        "DbiResourceId": "db-M5UMHA3BZF4IPX72DN7ZDEYVEQ",

        ...

        "CACertificateIdentifier": "rds-ca-2015",
        "StorageEncrypted": false,
        "DBInstanceClass": "db.m3.medium",
        "DbInstancePort": 3306,
        "DBInstanceIdentifier": "cc-mysql-staging-db"
    }
}

03 Repeat step no. 1 and 2 to downsize (resize) any other underused RDS instances available in the selected region.

04 Change the AWS region by updating the --region command parameter value and repeat the entire process for other regions.

Option 2: Suppress the rule check. If the selected underused RDS database instance configuration must remain unchanged (some workload scenarios can result in low resource utilization by design), you should turn off the conformity rule check for the specified instance from the Cloud Conformity console.

References

Publication date Mar 13, 2017