Use the Conformity Knowledge Base AI to help improve your Cloud Posture

vCPU-Based EC2 Instance Limit

Trend Cloud One™ – Conformity is a continuous assurance tool that provides peace of mind for your cloud infrastructure, delivering over 1000 automated best practice checks.

Risk Level: Medium (should be achieved)
Rule ID: EC2-011

Determine if the number of vCPUs (Virtual Central Processing Units) used by Amazon EC2 On-Demand instances per AWS region is close to the vCPU limit number established by AWS, and request a limit increase in order to avoid running into resource limitations during Amazon EC2 resource provisioning. With vCPU-based limits, Amazon EC2 measures usage towards each limit based on the total number of vCPUs that are assigned to the running On-Demand EC2 instances provisioned within your AWS cloud account. The following table shows the number of vCPUs provided for each instance size. The vCPU mapping for some EC2 instance types may differ – see AWS EC2 Instance Types for more details.

Instance Size vCPUs
nano 1
micro 1
small 1
medium 1
large 2
xlarge 4
2xlarge 8
3xlarge 12
4xlarge 16
8xlarge 32
9xlarge 36
10xlarge 40
12xlarge 48
16xlarge 64
18xlarge 72
24xlarge 96
32xlarge 128

This rule resolution is part of the Conformity solution.

Performance
efficiency

Monitoring vCPU-based limits for your On-Demand EC2 instances will help you to manage better your AWS compute power and avoid resource starvation in case your applications need to scale up fast, or in case you just need to provision multiple Amazon EC2 instances in a short period of time.

Note: Currently, there are 5 different vCPU-based limits for On-Demand instances: one limit that governs the usage of Standard Instance families such as A, C, D, H, I, M, R, T, and Z, one limit for Accelerated Instance family (F), one for graphic-intensive instances (G), one for general purpose GPU (P), one for special memory optimized (X) instances, and one for Machine Learning (Inf) instances. As an example, this conformity rule demonstrates how to check the vCPU-based limit (and increase the quota) for the Standard Instance family (i.e. A, C, D, H, I, M, R, T and Z instance types).


Audit

To determine if your AWS account is going to reach soon the vCPU-based limit set for the On-Demand instances, perform the following operations:

Using AWS Console

01 Sign in to the AWS Management Console.

02 Navigate to Amazon EC2 console at https://console.aws.amazon.com/ec2/.

03 In the navigation panel, under Instances, choose Instances.

04 Click inside the Filter instances box located under the console top menu, choose Instance type, and select one of the instance types available in the list, used in the current AWS region. This filtering technique will help you to determine how many On-Demand instances are currently provisioned for the selected instance type. Repeat this step for each instance type used for On-Demand instances within the current AWS region.

05 In the navigation panel select Limits to access the page with the vCPU-based instance limits set for the current AWS region.

06 Choose Calculate vCPU limit to open the simplified vCPU calculator necessary to compute the total vCPU limit requirements for your AWS cloud account.

07 On the Limits Calculator page, use the Add instance type button to add each instance type identified at step no. 4. Enter the number of EC2 instances available for each identified instance type in the Instance count box. Once all the instance types are added to the calculator, compare the value available in the vCPUs needed column (i.e. the total number of vCPUs in use) with the value available in the Current limit column (i.e. the vCPU limit quota set for the selected AWS region). If the total number of vCPUs in use is going to reach soon the limit quota set for the current AWS region, follow the instructions provided in the Remediation section to request a vCPU limit increase from Amazon Web Services (AWS).

08 Change the AWS cloud region from the console navigation bar and repeat the audit process for other regions.

Using AWS CLI

01 Run get-service-quota command (OSX/Linux/UNIX) with custom query filters to get the vCPU limit quota set for the On-Demand EC2 instances within the selected AWS region. The quota code used by Amazon Service Quotas for all the EC2 standard instances (instance type A, C, D, H, I, M, R, T and Z) is "L-1216C47A":

aws service-quotas get-service-quota
  --region us-east-1
  --service-code ec2
  --quota-code L-1216C47A
  --query 'Quota.Value'

02 The command output should return the vCPU limit quota configured for the selected AWS region:

32.0

03 Run describe-instances command (OSX/Linux/UNIX) with predefined and custom query filters to describe the instance type and the vCPU information for each running On-Demand EC2 instance provisioned in the selected AWS cloud region:

aws ec2 describe-instances
  --region us-east-1
  --filters "Name=instance-state-name,Values=running"
  --query 'Reservations[*].Instances[*].{"InstanceType": InstanceType,"CpuOptions": CpuOptions}'

04 The command output should return the vCPU information (the number of CPU cores per instance and the number of threads per CPU core) for each Amazon EC2 instance running in the selected AWS region:

[
	[
		{
			"InstanceType": "c5.4xlarge",
			"CpuOptions": {
				"CoreCount": 8,
				"ThreadsPerCore": 2
			}
		}
	],

	...

	[
		{
			"InstanceType": "c4.xlarge",
			"CpuOptions": {
				"CoreCount": 2,
				"ThreadsPerCore": 2
			}
		}
	]
]

Use the vCPU information returned by the describe-instances command output to determine the total number of vCPUs used in the selected AWS region. The number of vCPUs for an EC2 instance is the number of CPU cores ("CoreCount" attribute value) multiplied by the number of threads per core ("ThreadsPerCore" attribute value). Compare the total number of vCPUs used by the Amazon EC2 instances running in the selected region with the vCPU limit quota returned at step no. 2. If the total number of vCPUs in use is going to reach soon the limit quota set for the selected AWS region, follow the instructions provided in the Remediation section to request a vCPU limit increase from Amazon Web Services (AWS).

05 Change the AWS cloud region by updating the --region command parameter value and repeat the audit process for other regions.

Remediation / Resolution

To request an increase for the vCPU-based EC2 instance limit based on your workload requirements, perform the following operations:

Using AWS Console

01 Sign in to AWS Management Console.

02 Navigate to Amazon Service Quotas dashboard at https://console.aws.amazon.com/servicequotas/.

03 In the navigation panel, under Service Quotas, choose Dashboard.

04 Select Amazon Elastic Compute Cloud (Amazon EC2) to access the default quotas configured for the Amazon EC2 service.

05 Select All Standard (A, C, D, H, I, M, R, T, Z) Spot Instance Requests from the Service quotas, then choose Request quota increase to initiate the quota request.

06 In the Request quota increase: Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances configuration box, enter the total amount of vCPUs, representing the new quota limit, in the Change quota value box, then choose Request to send your vCPU quota increase request to Amazon Web Services (AWS).

07 Some quota increase requests create an AWS Support Center case. To track the status of your vCPU quota increase request, perform the following actions:

  1. Select Quota request history from the navigation panel, and click on the quota request link that you want to examine.
  2. On the Request quota increase: Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances panel, click on the Support Center case number ticket number to access the support case details (including case status) available for your request.

08 Change the AWS cloud region from the console navigation bar and repeat the remediation process for other regions.

Using AWS CLI

01 Run request-service-quota-increase command (OSX/Linux/UNIX) to request an increase for the number of vCPUs that can be used by On-Demand EC2 instances within the selected AWS region (i.e. vCPU-based instance limit). The quota code required by Amazon Service Quotas for the Amazon EC2 standard instances (instance type A, C, D, H, I, M, R, T and Z) is "L-1216C47A". Use the --desired-value command parameter to set the new quota limit based on your requirements:

aws service-quotas request-service-quota-increase
  --region us-east-1
  --service-code ec2
  --quota-code L-1216C47A
  --desired-value 96

02 The command output should return the new quota request configuration details:

{
	"RequestedQuota": {
		"QuotaName": "Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances",
		"Status": "PENDING",
		"DesiredValue": 96.0,
		"Created": 1571215789.939,
		"QuotaArn": "arn:aws:servicequotas:us-east-1:123456789012:ec2/L-1216C47A",
		"ServiceName": "Amazon Elastic Compute Cloud (Amazon EC2)",
		"GlobalQuota": false,
		"ServiceCode": "ec2",
		"QuotaCode": "L-1216C47A",
		"Requester": "{\"accountId\":\"123456789012\",\"callerArn\":\"arn:aws:sts::123456789012:assumed-role/ec2-manager/i-0abcdabcdabcdabcd\"}",
		"Id": "abcd1234abcd1234abcd1234abcd1234abcd1234",
		"Unit": "None"
	}
}

03 Some quota increase requests create automatically AWS Support Center cases. To retrieve the increase request status, run get-requested-service-quota-change command (OSX/Linux/UNIX) using the request ID returned at the previous step as the identifier parameter:

aws service-quotas get-requested-service-quota-change
  --region us-east-1
  --request-id abcd1234abcd1234abcd1234abcd1234abcd1234
  --query 'RequestedQuota.Status'

04 The command output should return the support case status (i.e. request status). If the request is pending the status should be set to "CASE_OPENED", otherwise the status should be "CASE_CLOSED":

"CASE_CLOSED"

05 Change the AWS cloud region by updating the --region command parameter value and repeat the remediation process for other regions.

References

Publication date Oct 21, 2019