- Knowledge Base
- Amazon Web Services
- Amazon Opensearch Service
- OpenSearch Domain In VPC
Ensure that your Amazon OpenSearch domains (clusters) are accessible only from Virtual Private Clouds (VPCs) for better flexibility and control over the cluster access using security groups, as this feature lets you keep all the traffic between your OpenSearch domains and VPC within the AWS network instead of going over the Internet.
This rule can help you with the following compliance standards:
- PCI
- APRA
- MAS
For further details on compliance standards supported by Conformity, see here.
This rule can help you work with the AWS Well-Architected Framework.
This rule resolution is part of the Conformity Security & Compliance tool for AWS.
Amazon OpenSearch domains that reside within a VPC have an extra layer of security when compared to OpenSearch domains that use public endpoints. Launching an Amazon OpenSearch cluster within a VPC network enables secure communication between the cluster (domain) and other AWS services without the need for an Internet Gateway, a NAT device or a VPN connection, and all the traffic remains secure within the AWS cloud.
Audit
To determine the access endpoint configuration implemented for your Amazon OpenSearch domains, perform the following actions:
Using AWS Console
01 Sign in to the AWS Management Console.
02 Navigate to Amazon OpenSearch console at https://console.aws.amazon.com/esv3/.
03 In the main navigation panel, under Dashboard, select Domains.
04 Select the OpenSearch domain that you want to examine and check the configuration value available in the Endpoint column. If the Endpoint is set to Internet, the selected Amazon OpenSearch domain is publicly accessible and does not reside within a Virtual Private Cloud (VPC).
05 Repeat step no. 4 for each Amazon OpenSearch cluster available within the current AWS region.
06 Change the AWS cloud region from the navigation bar and repeat the Audit process for other regions.
Using AWS CLI
01 Run list-domain-names command (OSX/Linux/UNIX) to list the name of each Amazon OpenSearch domain (cluster) available in the selected AWS region:
aws es list-domain-names --region us-east-1 --query 'DomainNames[*].DomainName'
02 The command output should return the identifier (name) of each OpenSearch domain provisioned in the selected region:
[ "trendmicro", "cloudconformity" ]
03 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the Amazon OpenSearch domain that you want to examine as the identifier parameter and custom query filters to describe the access endpoint URL generated for the selected domain:
aws es describe-elasticsearch-domain --region us-east-1 --domain-name trendmicro --query 'DomainStatus.Endpoint'
04 The command output should return the requested endpoint URL or null if the selected OpenSearch domain is currently associated with a VPC:
"search-trendmicro-abcd1234abcd1234abcd.us-east-1.es.amazonaws.com"
If the describe-elasticsearch-domain command output returns a public endpoint URL, as shown in the output example above, the selected Amazon OpenSearch domain is publicly accessible and does not reside within a Virtual Private Cloud (VPC).
05 Repeat steps no. 3 and 4 for each Amazon OpenSearch domain available in the selected AWS region.
06 Change the AWS cloud region by updating the --region command parameter value and repeat the Audit process for other regions.
Remediation / Resolution
To migrate your Amazon OpenSearch domain from public access to VPC-based access (recommended), you must re-create the domain with the necessary network access configuration. To relaunch your Amazon OpenSearch domain within a VPC and move the existing data, perform the following actions:
Using AWS CloudFormation
01 CloudFormation template (JSON):
{ "AWSTemplateFormatVersion": "2010-09-09", "Description": "Enable and Configure VPC-Based Access", "Resources": { "OpenSearchDomain": { "Type":"AWS::OpenSearchService::Domain", "Properties": { "DomainName": "cc-opensearch-domain", "EngineVersion": "OpenSearch_1.1", "ClusterConfig": { "InstanceType": "t3.small.search", "InstanceCount": "2" }, "EBSOptions": { "EBSEnabled": true, "VolumeType": "gp2", "VolumeSize": "50" }, "VPCOptions": { "SubnetIds": ["subnet-abcd1234", "subnet-1234abcd"], "SecurityGroupIds": ["sg-abcd1234abcd1234"] }, "AccessPolicies": { "Version":"2012-10-17", "Statement":[ { "Effect": "Deny", "Principal": { "AWS": "*" }, "Action":"es:*", "Resource": "arn:aws:es:us-east-1:123456789012:domain/cc-opensearch-domain/*" } ] } } } } }
02 CloudFormation template (YAML):
AWSTemplateFormatVersion: '2010-09-09' Description: Enable and Configure VPC-Based Access Resources: OpenSearchDomain: Type: AWS::OpenSearchService::Domain Properties: DomainName: cc-opensearch-domain EngineVersion: OpenSearch_1.1 ClusterConfig: InstanceType: t3.small.search InstanceCount: '2' EBSOptions: EBSEnabled: true VolumeType: gp2 VolumeSize: '50' VPCOptions: SubnetIds: - subnet-abcd1234 - subnet-1234abcd SecurityGroupIds: - sg-abcd1234abcd1234 AccessPolicies: Version: '2012-10-17' Statement: - Effect: Deny Principal: AWS: '*' Action: es:* Resource: arn:aws:es:us-east-1:123456789012:domain/cc-opensearch-domain/*
Using Terraform (AWS Provider)
01 Terraform configuration file (.tf):
terraform { required_providers { aws = { source = "hashicorp/aws" version = "~> 4.0" } } required_version = ">= 0.14.9" } provider "aws" { region = "us-east-1" } resource "aws_opensearch_domain" "opensearch-domain" { domain_name = "cc-opensearch-domain" engine_version = "OpenSearch_1.1" cluster_config { instance_type = "t3.small.search" instance_count = 2 } ebs_options { ebs_enabled = true volume_size = 50 volume_type = "gp2" } # Enable and Configure VPC-Based Access vpc_options { subnet_ids = ["subnet-abcd1234", "subnet-1234abcd"] security_group_ids = ["sg-abcd1234abcd1234"] } access_policies = <<POLICY { "Version": "2012-10-17", "Statement":[ { "Effect": "Deny", "Principal": { "AWS": "*" }, "Action":"es:*", "Resource": "arn:aws:es:us-east-1:123456789012:domain/cc-opensearch-domain/*" } ] } POLICY }
Using AWS Console
01 Sign in to the AWS Management Console.
02 Navigate to Amazon OpenSearch console at https://console.aws.amazon.com/esv3/.
03 In the main navigation panel, under Dashboard, select Domains.
04 Click on the name of the OpenSearch domain that you want to re-create and copy the domain configuration information.
05 Navigate back to the Domains page and choose Create domain to initiate the domain setup process.
06 On the Create domain setup page, perform the following actions:
- In the Name section, provide a unique name for your new OpenSearch domain in the Domain name box.
- In the Network section, select VPC access (recommended) under Network, to enable VPC-based access for the selected domain. Choose the VPC identifier from the VPC dropdown list, one or more subnets from the Subnets lists, and select one or multiple security groups from the Security groups dropdown list.
- Configure the rest of the domain settings using the configuration information copied at step no. 4.
- Choose Create to provision your new Amazon OpenSearch domain.
07 Once the new OpenSearch domain is created within the specified Virtual Private Cloud (VPC), upload the data from the source domain to the destination (new) domain.
08 (Optional) You can remove the publicly accessible (source) OpenSearch domain from your AWS account in order to avoid further charges. To delete the unneeded domain, perform the following actions:
- In the main navigation panel, under Dashboard, select Domains.
- Select the Amazon OpenSearch domain that you want to remove.
- Choose Delete from the console top menu to initiate the removal process.
- In the Delete domain? confirmation box, type the name of the resource in the required field, then choose Delete to confirm deletion.
09 Repeat steps no. 4 – 8 to enable VPC-based access for other OpenSearch domain available within the current AWS region.
10 Change the AWS cloud region from the navigation bar and repeat the Remediation process for other AWS regions.
Using AWS CLI
01 Run describe-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the Amazon OpenSearch domain that you want to re-create as the identifier parameter and custom query filters to describe the configuration information available for the selected domain:
aws es describe-elasticsearch-domain --region us-east-1 --domain-name trendmicro
02 The command output should return the configuration details available for the selected OpenSearch domain:
{ "DomainStatus": { "DomainId": "123456789012/trendmicro", "DomainName": "trendmicro", "ARN": "arn:aws:es:us-east-1:123456789012:domain/trendmicro", "Created": true, "Deleted": false, "Endpoint": "search-trendmicro-abcd1234abcdabcd1234abcd.us-east-1.es.amazonaws.com", "Processing": false, "UpgradeProcessing": false, "ElasticsearchVersion": "7.10", "ElasticsearchClusterConfig": { "InstanceType": "t3.small.elasticsearch", "InstanceCount": 2, "DedicatedMasterEnabled": false, "ZoneAwarenessEnabled": false, "WarmEnabled": false, "ColdStorageOptions": { "Enabled": false } }, "EBSOptions": { "EBSEnabled": true, "VolumeType": "gp2", "VolumeSize": 15 }, "AccessPolicies": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"es:*\",\"Resource\":\"arn:aws:es:us-east-1:123456789012:domain/trendmicro/*\"}]}", "SnapshotOptions": {}, "CognitoOptions": { "Enabled": false }, "EncryptionAtRestOptions": { "Enabled": true, "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/12341234-abcd-abcd-abcd-123412341234" }, "NodeToNodeEncryptionOptions": { "Enabled": false }, "AdvancedOptions": { "indices.fielddata.cache.size": "", "override_main_response_version": "false", "rest.action.multi.allow_explicit_index": "false" }, "ServiceSoftwareOptions": { "CurrentVersion": "R20211203-P2", "NewVersion": "", "UpdateAvailable": false, "Cancellable": false, "UpdateStatus": "COMPLETED", "Description": "There is no software update available for this domain.", "AutomatedUpdateDate": "2021-12-15T22:07:00+00:00", "OptionalDeployment": false }, "DomainEndpointOptions": { "EnforceHTTPS": false, "TLSSecurityPolicy": "Policy-Min-TLS-1-0-2019-07", "CustomEndpointEnabled": false }, "AdvancedSecurityOptions": { "Enabled": false, "InternalUserDatabaseEnabled": false }, "AutoTuneOptions": { "State": "ENABLE_IN_PROGRESS" } } }
03 Run create-elasticsearch-domain command (OSX/Linux/UNIX) using the configuration information returned at the previous step to relaunch the selected Amazon OpenSearch domain into a Virtual Private Cloud (VPC):
aws es create-elasticsearch-domain --region us-east-1 --domain-name trendmicro-v2 --elasticsearch-version 7.10 --elasticsearch-cluster-config InstanceType=t3.small.elasticsearch,InstanceCount=2 --ebs-options EBSEnabled=true,VolumeType=gp2,VolumeSize=50 --vpc-options SubnetIds=subnet-abcd1234,subnet-1234abcd,SecurityGroupIds=sg-abcd1234abcd1234 --access-policies '{"Version": "2012-10-17", "Statement": [ { "Effect": "Deny", "Principal": {"AWS": "*" }, "Action":"es:*", "Resource": "arn:aws:es:us-east-1:123456789012:domain/trendmicro/*" } ] }'
04 The command output should return the configuration metadata available for the new OpenSearch domain:
{ "DomainStatus": { "DomainId": "123456789012/trendmicro-v2", "DomainName": "trendmicro-v2", "ARN": "arn:aws:es:us-east-1:123456789012:domain/trendmicro-v2", "Created": true, "Deleted": false, "Processing": false, "UpgradeProcessing": false, "ElasticsearchVersion": "7.10", "ElasticsearchClusterConfig": { "InstanceType": "t3.small.elasticsearch", "InstanceCount": 2, "DedicatedMasterEnabled": false, "ZoneAwarenessEnabled": false, "WarmEnabled": false, "ColdStorageOptions": { "Enabled": false } }, "EBSOptions": { "EBSEnabled": true, "VolumeType": "gp2", "VolumeSize": 15 }, "AccessPolicies": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"es:*\",\"Resource\":\"arn:aws:es:us-east-1:123456789012:domain/trendmicro/*\"}]}", "SnapshotOptions": {}, "VPCOptions": { "SubnetIds": [ "subnet-abcd1234", "subnet-1234abcd" ], "SecurityGroupIds": [ "sg-abcd1234abcd1234" ] }, "CognitoOptions": { "Enabled": false }, "EncryptionAtRestOptions": { "Enabled": true, "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/abcd1234-abcd-1234-abcd-1234abcd1234" }, "NodeToNodeEncryptionOptions": { "Enabled": false }, "AdvancedOptions": { "indices.fielddata.cache.size": "", "override_main_response_version": "false", "rest.action.multi.allow_explicit_index": "false" }, "ServiceSoftwareOptions": { "CurrentVersion": "R20211203-P2", "NewVersion": "", "UpdateAvailable": false, "Cancellable": false, "UpdateStatus": "COMPLETED", "Description": "There is no software update available for this domain.", "AutomatedUpdateDate": "2021-12-15T22:07:00+00:00", "OptionalDeployment": false }, "DomainEndpointOptions": { "EnforceHTTPS": false, "TLSSecurityPolicy": "Policy-Min-TLS-1-0-2019-07", "CustomEndpointEnabled": false }, "AdvancedSecurityOptions": { "Enabled": false, "InternalUserDatabaseEnabled": false }, "AutoTuneOptions": { "State": "ENABLE_IN_PROGRESS" } } }
05 Once the new OpenSearch domain is launched within the specified Virtual Private Cloud (VPC), upload the data from the source domain to the destination (new) domain.
06 (Optional) You can remove the publicly accessible (source) OpenSearch domain from your AWS account in order to avoid further charges. To shut it down, run delete-elasticsearch-domain command (OSX/Linux/UNIX) using the name of the OpenSearch domain that you want to delete as the identifier parameter:
aws es delete-elasticsearch-domain --region us-east-1 --domain-name trendmicro
07 The command output should return the configuration metadata of the removed OpenSearch domain:
{ "DomainStatus": { "DomainId": "123456789012/trendmicro", "DomainName": "trendmicro", "ARN": "arn:aws:es:us-east-1:123456789012:domain/trendmicro", "Created": true, "Deleted": false, "Endpoint": "search-trendmicro-abcd1234abcdabcd1234abcd.us-east-1.es.amazonaws.com", "Processing": false, "UpgradeProcessing": false, "ElasticsearchVersion": "7.10", "ElasticsearchClusterConfig": { "InstanceType": "t3.small.elasticsearch", "InstanceCount": 2, "DedicatedMasterEnabled": false, "ZoneAwarenessEnabled": false, "WarmEnabled": false, "ColdStorageOptions": { "Enabled": false } }, "EBSOptions": { "EBSEnabled": true, "VolumeType": "gp2", "VolumeSize": 15 }, "AccessPolicies": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Deny\",\"Principal\":{\"AWS\":\"*\"},\"Action\":\"es:*\",\"Resource\":\"arn:aws:es:us-east-1:123456789012:domain/trendmicro/*\"}]}", "SnapshotOptions": {}, "CognitoOptions": { "Enabled": false }, "EncryptionAtRestOptions": { "Enabled": true, "KmsKeyId": "arn:aws:kms:us-east-1:123456789012:key/12341234-abcd-abcd-abcd-123412341234" }, "NodeToNodeEncryptionOptions": { "Enabled": false }, "AdvancedOptions": { "indices.fielddata.cache.size": "", "override_main_response_version": "false", "rest.action.multi.allow_explicit_index": "false" }, "ServiceSoftwareOptions": { "CurrentVersion": "R20211203-P2", "NewVersion": "", "UpdateAvailable": false, "Cancellable": false, "UpdateStatus": "COMPLETED", "Description": "There is no software update available for this domain.", "AutomatedUpdateDate": "2021-12-15T22:07:00+00:00", "OptionalDeployment": false }, "DomainEndpointOptions": { "EnforceHTTPS": false, "TLSSecurityPolicy": "Policy-Min-TLS-1-0-2019-07", "CustomEndpointEnabled": false }, "AdvancedSecurityOptions": { "Enabled": false, "InternalUserDatabaseEnabled": false }, "AutoTuneOptions": { "State": "ENABLE_IN_PROGRESS" } } }
08 Repeat steps no. 1 – 7 to enable VPC-based access for other OpenSearch domain available in the selected AWS region.
09 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 12 to perform the Remediation process for other regions.
References
- AWS Documentation
- Amazon OpenSearch Service FAQs
- Amazon Elasticsearch Service announces support for Amazon Virtual Private Cloud (VPC)
- Launching your Amazon OpenSearch Service domains within a VPC
- Step 2: Upload data to Amazon OpenSearch Service for indexing
- AWS Command Line Interface (CLI) Documentation
- es
- list-domain-names
- describe-elasticsearch-domain
- create-elasticsearch-domain
- CloudFormation Documentation
- Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) resource type reference
- Terraform Documentation
- AWS Provider