Use the Conformity Knowledge Base AI to help improve your Cloud Posture

Redshift Reserved Node Coverage

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)

Ensure that the reservation coverage percentage (i.e. the percentage of node instance hours that a reservation covered) available for your Amazon Redshift Reserved Nodes (RNs) does not fall below a defined threshold. Reserved Node (RN) coverage tracks the number of running node hours that are covered by reservations, and can be measured over a daily, monthly, quarterly, or yearly cadence. For example, you can monitor your RN coverage either at an aggregate level (e.g. monthly coverage of your entire Amazon Redshift RN fleet) or at a more granular level of detail (e.g. monthly coverage of Amazon Redshift ds2.xlarge nodes running within US West - Oregon region). The reservation coverage threshold (percentage) and the time period to retrieve the reservation coverage data for, must be defined in the conformity rule settings, on the Trend Cloud One™ – Conformity account console.

This rule can help you work with the AWS Well-Architected Framework.

Cost
optimisation

Reserved Nodes (RNs) represent a solid strategy to cut down on Amazon Redshift costs by taking advantage of AWS billing at a discounted hourly rate. The Reserved Node coverage report offered by Amazon Cost Management service allows you to discover how much of your overall Redshift usage is covered by RNs, so that you can make informed decisions about when to purchase or modify a Reserved Node to ensure maximum reservation coverage. The Cost Management service provides effective recommendations for which reservations to purchase in order to help you reduce your Amazon Redshift costs.


Audit

To determine if the reservation coverage (percentage) of your Amazon Redshift Reserved Nodes fall below your target coverage, perform the following operations:

Using AWS Console

01 Sign in to your Trend Cloud One™ – Conformity account, access Redshift Reserved Node Coverage conformity rule settings and identify the Amazon Redshift reservation coverage threshold (percentage) defined for the specified period of time, configured for your AWS cloud account.

02 Sign in to AWS Management Console.

03 Navigate to Amazon Cost Management console at https://console.aws.amazon.com/cost-management/home.

04 In the navigation panel, under Reservations, choose Coverage report to access the reservation coverage report provided by Amazon Cost Management service for your AWS account. This helps you identify opportunities for cost savings using Redshift Reserved Nodes (RNs).

05 On the Coverage report page, perform the following actions:

  1. For FILTERS, select Service, Redshift, and choose Apply filters to display the RN coverage data for Amazon Redshift nodes only.
  2. Click on the time period dropdown list to access the date/time picker, and choose the time frame to retrieve data about your reservation coverage (daily, monthly, quarterly or yearly) as specified in the conformity rule settings, identified at step no. 1. Once the appropriate time frame is selected, choose Apply to show the reservation coverage for that period.
  3. Check the reservation coverage (percentage) displayed in the Average Coverage (hours) panel box, under RI Coverage. If the reservation coverage (percentage) is lower than the target coverage threshold identified at step no. 1, the reservation coverage for your Amazon Redshift Reserved Nodes (RNs) falls below your target coverage, therefore actions must be taken to ensure maximum coverage.

06 If required, repeat step no. 1 – 5 determine the Amazon Redshift reservation coverage for other AWS accounts.

Using AWS CLI

01 Sign in to your Trend Cloud One™ – Conformity account, access Redshift Reserved Node Coverage conformity rule settings and identify the Amazon Redshift reservation coverage threshold (percentage) defined for the specified period of time, configured for your AWS cloud account.

02 Define the AWS service for which to return the reservation coverage, i.e. Amazon Redshift. Save the following configuration document to a JSON file named aws-service-filter.json:

{
  "Dimensions": {
     "Key": "SERVICE",
     "Values": [ "Amazon Redshift" ]
  }
}

03 Run get-reservation-coverage command (OSX/Linux/UNIX) using the filter configuration document created at the previous step as value for the --filter parameter and custom query filters to retrieve the reservation coverage for your Amazon Redshift Reserved Nodes (RNs), for the specified period of time. The --time-period parameter value indicates the start and end dates of the period that you want to retrieve data about reservation coverage for, and must match the time frame defined for the target threshold, identified at step no. 1:

aws ce get-reservation-coverage
	--time-period Start=2020-10-25,End=2020-11-25
	--granularity MONTHLY
	--filter file://aws-service-filter.json
	--query 'CoveragesByTime[*].Total.CoverageHours.CoverageHoursPercentage'

04 The command output should return the requested reservation coverage (percentage):

[
    "71"
]

If the reservation coverage (percentage) returned by the get-reservation-coverage command output is lower than the target coverage threshold identified at step no. 1, the reservation coverage for your Amazon Redshift Reserved Nodes (RNs) falls below your target coverage, therefore actions must be taken to ensure maximum coverage.

05 If required, repeat step no. 1 – 4 determine the Amazon Redshift reservation coverage for other AWS accounts.

Remediation / Resolution

To ensure maximum reservation coverage for your Amazon Redshift Reserved Nodes (RNs), follow the context-aware RN purchase recommendations provided by Amazon Cost Management service to optimize your AWS costs by performing the following operations:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Amazon Cost Management console at https://console.aws.amazon.com/cost-management/home.

03 In the navigation panel, under Reservations, choose Recommendations to access the Reserved Node (RN) purchase recommendations provided by Amazon Cost Management. These recommendations are based on your past usage and indicate opportunities to maximize your cost savings as compared to On-Demand usage.

04 On the Recommendations page, perform the following actions:

  1. For Select recommendation type, choose Redshift to show purchase recommendations for Amazon Redshift nodes only.
  2. For RI Recommendation Parameters, select the required RN attributes and adjust the usage time period for which your purchase recommendations should be calculated:
    • For RI term, select the length of the RN reservation that you want recommendations for.
    • For Payment option, choose whether or not you want to pay for recommended Reserved Nodes upfront.
    • For Based on the past, select the number of previous days that you want AWS to consider when it calculates your RN recommendations.
  3. After all the required parameters are provided, you can see all the Reserved Node (RN) purchase recommendations made by Amazon Cost Management in the Purchase Recommendations section. Cost Management service recommends the smallest size node available in a node family. This makes it easier to purchase a size-flexible Reserved Nodes. The service also shows the equal number of normalized units so that you can purchase any node size that you want. For example, your RN purchase recommendation would be for ds2.xlarge because that is the smallest size node in the Dense Storage DS2 family.

05 To adopt the RN recommendations provided by Amazon Cost Management service at the previous step and purchase new Redshift Reserved Nodes (RNs), navigate to Amazon Redshift console at https://console.aws.amazon.com/redshift.

06 In the navigation panel, under Redshift dashboard, choose Reserved nodes.

07 Click Purchase Reserved Nodes button from the console top menu to initiate the purchasing process.

08 On the Purchase Reserved Nodes panel, specify the necessary Reserved Node (RN) offering details based on the purchase recommendations provided by Cost Management service at step no 4c:

  1. For Node Type, select the required Amazon Redshift node type.
  2. For Term, select the length of time you want the Redshift node reserved.
  3. For Offering Type, choose the payment type that you want to use for your reservation (e.g. All Upfront).
  4. For Number of Nodes, specify the number of Amazon Redshift nodes that you want to reserve with this order.
  5. After you provide all the parameters required for Reserved Node purchase, the Redshift service should display the pricing information. Choose Continue.
  6. On the purchase summary panel, review your reserved node(s) configuration information. If the information is correct, choose Order to purchase the reserved Amazon Redshift node(s). The order processing may take few minutes as the node(s) status will change from pending-payment to active.

09 If required, repeat step no. 7 and 8 to purchase more Amazon Redshift Reserved Nodes as specified in the RN purchase recommendations list provided by Amazon Cost Management.

10 If required, repeat step no. 1 – 9 to perform the remediation process for other AWS accounts.

Using AWS CLI

01 Run get-reservation-purchase-recommendation command (OSX/Linux/UNIX) with custom query filters to get recommendations for which Amazon Redshift Reserved Nodes to purchase. These recommendations are based on your past usage and indicate opportunities to maximize your cost savings as compared to On-Demand usage. The following command request example retrieves purchase recommendations for All Upfront Redshift nodes with one-year term, based on the last 30 days of Amazon Redshift usage:

aws ce get-reservation-purchase-recommendation
	--service "Amazon Redshift"
	--lookback-period-in-days THIRTY_DAYS
	--term-in-years ONE_YEAR
	--payment-option ALL_UPFRONT
	--query 'Recommendations[*].RecommendationDetails[]'

02 The command output should return the Amazon Redshift RN purchase recommendation(s) provided by Amazon Cost Management service for your AWS cloud account. AWS recommends the smallest size node in a node family. This makes it easier to purchase a size-flexible Reserved Node. For example, the following command output describes one RN purchase recommendation for ds2.xlarge nodes:

[
    {
        "AverageUtilization": "100.0000000000",
        "RecurringStandardMonthlyCost": "0.0000000000",
        "MinimumNumberOfNodesUsedPerHour": "2.0000000000",
        "NodeDetails": {
            "RedshiftInstanceDetails": {
                "Family": "ds2",
                "SizeFlexEligible": true,
                "CurrentGeneration": true,
                "Region": "Asia Pacific (Sydney)",
                "NodeType": "ds2.xlarge"
            }
        },
        "EstimatedMonthlySavingsAmount": "570.9558887800",
        "AverageNumberOfNodesUsedPerHour": "2.0000000000",
        "EstimatedBreakEvenInMonths": "183.0568700700",
        "CurrencyCode": "USD",
        "EstimatedMonthlySavingsPercentage": "40.0000",
        "UpfrontCost": "6435.0000000000",
        "MaximumNormalizedUnitsUsedPerHour": "1.0000000000",
        "MaximumNumberOfNodesUsedPerHour": "2.0000000000",
        "RecommendedNumberOfNodesToPurchase": "2",
        "EstimatedMonthlyOnDemandCost": "456.2750000000",
        "EstimatedReservationCostForLookbackPeriod": "181.1780820000",
        "RecommendedNormalizedUnitsToPurchase": "1.0",
        "AverageNormalizedUnitsUsedPerHour": "1.0000000000",
        "AccountId": "123456789012",
        "MinimumNormalizedUnitsUsedPerHour": "1.0000000000"
    }
]

03 Run describe-reserved-node-offerings command (OSX/Linux/UNIX) to describe the pricing details and information about the available Amazon Redshift Reserved Node offerings, based on the RN purchase recommendations provided by Amazon Cost Management at the previous step:

aws redshift describe-reserved-node-offerings
	--region ap-southeast-2
	--query 'ReservedNodeOfferings[?(NodeType==`ds2.xlarge`) && (Duration==`31536000`) && (OfferingType==`All Upfront`)]'

04 The command output should return the requested offering information (including the Reserved Node offering ID – highlighted):

[
    {
        "OfferingType": "All Upfront",
        "FixedPrice": 6435.0,
        "NodeType": "ds2.xlarge",
        "UsagePrice": 0.0,
        "RecurringCharges": [],
        "ReservedNodeOfferingType": "Regular",
        "Duration": 31536000,
        "ReservedNodeOfferingId": "abcd1234-abcd-1234-abcd-1234abcd1234",
        "CurrencyCode": "USD"
    }
]

05 Run purchase-reserved-node-offering command (OSX/Linux/UNIX) using the ID of the RN offering that you want to use, to purchase Amazon Redshift Reserved Nodes based on the offering information returned at the previous step:

aws redshift purchase-reserved-node-offering
 	--region ap-southeast-2
 	--reserved-node-offering-id abcd1234-abcd-1234-abcd-1234abcd1234
 	--node-count 2

06 The command output should return the metadata available for your new Amazon Redshift Reserved Nodes (RNs):

{
   "ReservedNode": {
      "OfferingType": "All Upfront",
      "FixedPrice": 6435.0,
      "NodeType": "ds2.xlarge",
      "ReservedNodeId": "aaaabbbb-cccc-aaaa-bbbb-aaaabbbbcccc",
      "UsagePrice": 0.0,
      "RecurringCharges": [
         {
            "RecurringChargeAmount": 0.478,
            "RecurringChargeFrequency": "Hourly"
         }
      ],
      "NodeCount": 2,
      "State": "payment-pending",
      "StartTime": "2020-11-18T15:00:00.000Z",
      "Duration": 31536000,
      "ReservedNodeOfferingId": "abcd1234-abcd-1234-abcd-1234abcd1234"
   },
   "ResponseMetadata": {
      "RequestId": "1234abcd-1234-abcd-1234-abcd1234abcd"
   }
}

07 If required, repeat steps no. 3 – 6 to purchase more Amazon Redshift Reserved Nodes as specified in the RN purchase recommendations list provided by Amazon Cost Management.

08 If required, repeat step no. 1 – 7 to perform the entire remediation process for other AWS accounts.

References

Publication date Nov 23, 2020