Redshift Disk Space Usage

01 Sign in to AWS Management Console.

02 Navigate to Redshift dashboard at https://console.aws.amazon.com/redshift/.

03 In the left navigation panel, under Redshift Dashboard, click Clusters.

04 Choose the Redshift cluster that you want to examine then click on its identifier/name link, listed in the Cluster column.

05 Click on Performance tab from the dashboard top menu to access the cluster monitoring panel.

06 On the Redshift cluster monitoring panel, set the following parameters:

• From the Time Range dropdown list, select Last 24 Hours.
• From the Period list, select 1 Hour.
• From the Statistic dropdown list, select Maximum.
• From the Metrics dropdown list, select PercentageDiskSpaceUsed.

07 Repeat steps no. 4 –6 to verify the disk space usage details for other Redshift clusters provisioned in the current region.

08 Change the AWS region from the navigation bar and repeat the audit process for other regions.

01 Run describe-clusters command (OSX/Linux/UNIX) using custom query filters to list the IDs of all AWS Redshift clusters created in the selected region:

aws redshift describe-clusters
	--region us-east-1
	--output table
	--query 'Clusters[*].ClusterIdentifier'

02 The command output should return a table with the requested cluster IDs:

--------------------------
|    DescribeClusters    |
+------------------------+
|  cc-prod-data-cluster  |
|  cc-sandbox-db-cluster |
+------------------------+

03 Run get-metric-statistics command (OSX/Linux/UNIX) to get the statistics recorded by AWS CloudWatch for the PercentageDiskSpaceUsed metric representing the percent of disk space used by the selected Redshift cluster. The following command example returns the percent of disk space used by an AWS Redshift cluster identified by the name cc-prod-data-cluster, usage data captured during a 24 hours time frame, using 1 hour time range as the granularity of the returned datapoints:

aws cloudwatch get-metric-statistics
	--region us-east-1
	--metric-name PercentageDiskSpaceUsed
	--start-time 2017-09-11T11:05:41
	--end-time 2017-09-12T11:05:41
	--period 3600
	--namespace AWS/Redshift
	--statistics Maximum
	--dimensions="Name=ClusterIdentifier,Value=cc-prod-data-cluster"

04 The command output should return the disk usage details requested:

{
    "Datapoints": [
        {
            "Timestamp": "2017-09-11T11:05:41Z",
            "Maximum": 96.2085,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-09-11T12:05:41Z",
            "Maximum": 96.2085,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-09-11T13:05:41Z",
            "Maximum": 96.2085,
            "Unit": "Percent"
        },

        ...

        {
            "Timestamp": "2017-09-12T09:05:41Z",
            "Maximum": 97.52783333333333333,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-09-12T10:05:41Z",
            "Maximum": 97.52783333333333333,
            "Unit": "Percent"
        },
        {
            "Timestamp": "2017-09-12T11:05:41Z",
            "Maximum": 97.52783333333333333,
            "Unit": "Percent"
        }
    ],
    "Label": "PercentageDiskSpaceUsed"
}

05 Repeat step no. 3 and 4 to check the disk space usage details for other Redshift clusters provisioned in the current region.

06 Change the AWS region by updating the --region command parameter value and repeat steps no. 1 - 5 to perform the entire audit process for other regions.

01 Sign in to AWS Management Console.

02 Navigate to Redshift dashboard at https://console.aws.amazon.com/redshift/.

03 In the left navigation panel, under Redshift Dashboard, click Clusters.

04 Choose the Redshift cluster that you want to resize (see Audit section part I to identify the right resource), then click on its identifier/name link, listed in the Cluster column.

05 On the selected cluster configuration page, click the Cluster dropdown button from the dashboard top menu and select Resize.

06 Within Resize Cluster configuration window, perform the following:

1. Select Multi Node from the Cluster type dropdown list to configure the selected cluster into a multi-node cluster.
2. Increase the number of nodes available within Number of nodes box.
3. Click Resize to apply the changes and resize the selected Redshift cluster. The Cluster Status should change now to resizing. IMPORTANT: When you start the resize operation, the Redshift service puts the existing cluster into read-only mode until the resize is complete. During this time, you can only run queries that read from the cluster database and you cannot run any queries that write to the database (including read-write queries).

07 Repeat steps no. 4 – 6 to increase the storage capacity for other Amazon Redshift clusters with high disk space usage, provisioned in the current region.

08 Change the AWS region from the navigation bar and repeat the remediation/resolution process for other regions.

01 Run modify-cluster command (OSX/Linux/UNIX) using the name of the cluster that you want to resize as identifier (see Audit section part II to identify the right Redshift resource) to resize (scale-out) the selected Redshift cluster by adding more nodes to its configuration. IMPORTANT: When you execute the modify-cluster command and start the resize operation, the AWS Redshift service puts the existing cluster into read-only mode until the resize is complete. During this time, you can only run queries that read from the cluster database and you cannot run any queries that write to the database (including read-write queries):

aws redshift modify-cluster
	--region us-east-1
	--cluster-identifier cc-prod-data-cluster
	--cluster-type multi-node
	--number-of-nodes 2
	--node-type ds1.xlarge

02 If successful, the command output should return the new configuration metadata for the selected AWS Redshift cluster:

{
    "Cluster": {
        "PubliclyAccessible": true,
        "VpcSecurityGroups": [
            {
                "Status": "active",
                "VpcSecurityGroupId": "sg-04d8d412"
            }
        ],
        "NumberOfNodes": 1,
        "PendingModifiedValues": {
            "NodeType": "ds1.xlarge",
            "ClusterType": "multi-node",
            "NumberOfNodes": 2
        },
        "VpcId": "vpc-3de5f538",
        "ClusterVersion": "1.0",
        "AutomatedSnapshotRetentionPeriod": 7,

        ...

        "ClusterParameterGroups": [
            {
                "ParameterGroupName": "default.redshift-1.0",
                "ParameterApplyStatus": "in-sync"
            }
        ],
        "PreferredMaintenanceWindow": "fri:08:30-fri:09:00",
        "IamRoles": [],
        "AllowVersionUpgrade": true,
        "ClusterCreateTime": "2017-08-10T08:38:15.504Z",
        "ClusterSubnetGroupName": "default",
        "EnhancedVpcRouting": false,
        "ClusterSecurityGroups": [],
        "ClusterIdentifier": "cc-prod-data-cluster",
        "AvailabilityZone": "us-east-1a",
        "NodeType": "ds1.xlarge",
        "Encrypted": false,
        "ClusterStatus": "resizing"
    }
}

03 Repeat step no. 1 and 2 to increase the storage capacity for other Amazon Redshift clusters with high disk space usage, provisioned in the current region.

04 Change the AWS region by updating the --region command parameter value and repeat the entire process for other regions.