Check for Missing Model Customization Job Security Groups

01 Sign in to the AWS Management Console.

02 Navigate to Amazon Bedrock console available at https://console.aws.amazon.com/bedrock/.

03 In the main navigation panel, under Foundation models, select Custom models.

04 Select the Jobs tab to list the Amazon Bedrock model customization jobs available in the current AWS region.

05 Click on the name (link) of the model customization job that you want to examine (fine-tuned or continued pre-training job), available in the Job name column.

06 In the VPC Settings section, click on the ID (link) of the security group associated with the customization job, listed under Security Group(s). If the following error message is returned: An error occurred. The security group '\<security-group-id\>' does not exist, the security group associated with the selected Amazon Bedrock model customization job is no longer available.

07 Repeat steps no. 6 for each security group associated with the selected model customization job.

08 Repeat steps no. 5 - 7 for each model customization job available within the current AWS region.

09 Change the AWS cloud region from the navigation bar to repeat the Audit process for other regions.

01 Run list-model-customization-jobs command (OSX/Linux/UNIX) to list the name of each Amazon Bedrock model customization job (fine-tuned or continued pre-training job), available in the selected AWS cloud region:

aws bedrock list-model-customization-jobs
  --region us-east-1
  --query 'modelCustomizationJobSummaries[*].jobName'

02 The command output should return the requested model customization job names:

[
	"tm-fine-tuned-model-training-job",
	"tm-continued-pre-training-job"
]

03 Run get-model-customization-job command (OSX/Linux/UNIX) with the name of the Amazon Bedrock model customization job that you want to examine as the identifier parameter and custom output filters to describe the ID(s) of the security group(s) associated with the selected model customization job:

aws bedrock get-model-customization-job
  --region us-east-1
  --job-identifier tm-fine-tuned-model-training-job
  --query 'vpcConfig.securityGroupIds'

04 The command output should return the requested security group ID(s):

[
	"sg-01234abcd1234abcd",
	"sg-0abcdabcdabcdabcd"
]

05 Run describe-security-groups command (OSX/Linux/UNIX) using the ID of the security group returned at the previous step as the identifier parameter, to describe the configuration information available for the specified resource:

aws ec2 describe-security-groups
  --region us-east-1
  --group-names sg-01234abcd1234abcd

06 The command output should return the requested configuration information if the security group is still available or an error message otherwise:

An error occurred (InvalidGroup.NotFound) when calling the DescribeSecurityGroups operation: The security group 'sg-01234abcd1234abcd' does not exist in default VPC 'vpc-0abcd1234abcd1234'

07 Repeat steps no. 5 and 6 for each security group associated with the selected model customization job.

08 Repeat steps no. 3 - 7 for each model customization job available in the selected AWS region.

09 Change the AWS cloud region by updating the --region command parameter value and repeat steps no. 1 – 8 to perform the Audit process for other regions.

01 Sign in to the AWS Management Console.

02 Navigate to VPC dashboard available at https://console.aws.amazon.com/vpc.

03 In the left navigation panel, under Security, choose Security groups.

04 Choose Create security group and perform the following actions:

1. In the Security group name box, enter a unique name for your new custom security group.
2. In the Description box, provide a short description to reflect the security group usage.
3. From the VPC dropdown list, select the VPC network in which to create the security group.
4. In the Inbound rules section, choose Add rule to configure the inbound/ingress rule(s) required to control traffic between Amazon Bedrock and VPC resources, based on your application requirements.
5. In the Outbound rules section, choose Add rule to configure the outbound rule(s) required to control egress traffic.
6. (Optional) For Tags – optional, choose Add new tag to create and apply user-defined tags to the new security group.
7. Choose Create security group to create your new VPC security group.

05 Navigate to Amazon Bedrock console available at https://console.aws.amazon.com/bedrock/.

06 In the main navigation panel, under Foundation models, select Custom models.

07 Select the Jobs tab to list the Amazon Bedrock model customization jobs available in the current AWS region.

08 Click on the name (link) of the model customization job that you want to re-create (i.e. source job) and note the job configuration information such as base (source) model name, input (training) and output data config, and any defined hyperparameters.

09 Navigate back to the Custom models page, choose the Jobs tab, and perform the following actions to create your new model customization job:

10 For fine-tuned jobs, choose Create Fine-tuning job, and perform the following operations:

1. For Model details, select the base (source) model that you want to customize and provide a unique name for your new fine-tuned model. Check the Model encryption checkbox and select the ID of your Amazon KMS Customer Managed Key (CMK), from the Choose an AWS KMS key list. (Optional) For Tags - optional, create any required tag sets, according to the source job tagging scheme.
2. For Job configuration, provide a unique name for your new model customization job in the Job name box. (Optional) For Tags - optional, create any required tag sets, according to the source job tagging scheme.
3. For VPC settings - optional, select the Virtual Private Cloud (VPC) that defines the virtual networking environment for your new job, choose a VPC subnet for each Availability Zone in the selected region, and select the security group required for controlling traffic between Amazon Bedrock and VPC resources, created at step no. 4.
4. For Input data, specify the S3 location of your training input and validation datasets.
5. For Hyperparameters, specify the hyperparameter values that you want to use for your new fine-tuning job (must match the source job hyperparameters).
6. For Output data, choose Amazon S3 location to store the model validation outputs.
7. Bedrock model customization jobs require permissions to write to Amazon S3 on your behalf. For Service access, choose Create and use a new service role under Choose a method to authorize Bedrock, and provide a unique name for the service role in the Service role name box. Choose View permission details to view the permission policy and trust relationship policy created for the new IAM role.
8. Choose Create Fine-tuning job to create your new, compliant Amazon Bedrock fine-tuning job.

11 For continued pre-training jobs, choose Create Continued Pre-training jobs, and perform the following actions:

1. For Model details, select the base (source) model that you want to customize and provide a unique name for your new pre-trained model. Check the Model encryption checkbox and select the ID of your Amazon KMS Customer Managed Key (CMK), from the Choose an AWS KMS key list. (Optional) For Tags - optional, create any required tag sets, according to the source job tagging scheme.
2. For Job configuration, provide a unique name for your new model customization job in the Job name box. (Optional) For Tags - optional, create any required tag sets, according to the source job tagging scheme.
3. For VPC settings - optional, select the Virtual Private Cloud (VPC) that defines the virtual networking environment for your new job, choose a VPC subnet for each Availability Zone (AZ) in the selected region, and select the security group created at step no. 4.
4. For Input data, specify the S3 location of your training input and validation datasets.
5. For Hyperparameters, specify the hyperparameter values that you want to use for your new pre-training job (must match the source job hyperparameters).
6. For Output data, choose Amazon S3 location to store the model validation outputs.
7. For Service access, choose Create and use a new service role under Choose a method to authorize Bedrock, and provide a unique name for the service role in the Service role name box. Choose View permission details to view the permission policy and trust relationship policy created for the new IAM role.
8. Choose Create Continued Pre-training jobs to create your new, compliant Amazon Bedrock continued pre-training job.

12 Repeat steps no. 2 - 11 for each model customization job that you want to re-create, available within the current AWS region.

13 Change the AWS cloud region from the navigation bar and repeat the Remediation process for other regions.

01 Run create-security-group command (OSX/Linux/UNIX) to re-create the missing security group required by your Amazon Bedrock model customization job. Ensure that the new security group and the selected customization job share the same VPC network:

aws ec2 create-security-group
  --region us-east-1
  --group-name tm-training-job-security-group
  --description "Security Group for Amazon Bedrock Model Customization Jobs"
  --vpc-id vpc-0abcd1234abcd1234

02 The command output should return the ID of the new VPC security group:

{
	"GroupId": "sg-01234abcd1234abcd"
}

03 Run authorize-security-group-ingress command (OSX/Linux/UNIX) with the ID of the security group returned at the previous step as the identifier parameter, to create the inbound rule required to control traffic between Amazon Bedrock and VPC resources, based on your application requirements:

aws ec2 authorize-security-group-ingress
  --region us-east-1
  --group-id sg-01234abcd1234abcd
  --protocol tcp
  --port 8080
  --cidr 0.0.0.0/0

04 The command output should return the configuration information available for the new inbound rule:

{
	"Return": true,
	"SecurityGroupRules": [
		{
			"SecurityGroupRuleId": "sgr-01234abcd1234abcd",
			"GroupId": "sg-01234abcd1234abcd",
			"GroupOwnerId": "123456789012",
			"IsEgress": false,
			"IpProtocol": "tcp",
			"FromPort": 8080,
			"ToPort": 8080,
			"CidrIpv4": "0.0.0.0/0"
		}
	]
}

05 Run authorize-security-group-egress command (OSX/Linux/UNIX) with the ID of the newly created security group as the identifier parameter to configure the necessary outbound rule, based on your application requirements:

aws ec2 authorize-security-group-egress
  --region us-east-1
  --group-id sg-01234abcd1234abcd
  --ip-permissions '[{"IpProtocol": "tcp", "FromPort": 8080, "ToPort": 8080, "IpRanges": [{"CidrIp": "0.0.0.0/0"}]}]'

06 The command output should return the configuration information available for the new outbound rule:

{
	"Return": true,
	"SecurityGroupRules": [
		{
			"SecurityGroupRuleId": "sgr-01234abcd1234abcd",
			"GroupId": "sg-01234abcd1234abcd",
			"GroupOwnerId": "123456789012",
			"IsEgress": true,
			"IpProtocol": "tcp",
			"FromPort": 8080,
			"ToPort": 8080,
			"CidrIpv4": "0.0.0.0/0"
		}
	]
}

07 Run create-model-customization-job command (OSX/Linux/UNIX) to re-create your Amazon Bedrock model customization job with the necessary VPC network configuration. The following example creates a fine-tuning job to customize a base model identified by the following Amazon Resource Name (ARN): "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-text-lite-v1:0:4k", with a security group identified by the ID: "sg-01234abcd1234abcd":

aws bedrock create-model-customization-job
  --region us-east-1
  --job-name tm-new-fine-tuned-model-training-job
  --custom-model-name tm-new-fine-tuned-model
  --base-model-identifier arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-text-lite-v1:0:4k
  --role-arn arn:aws:iam::123456789012:role/service-role/tm-bedrock-new-service-role
  --training-data-config s3Uri="s3://tm-bedrock-input-data/train.jsonl"
  --output-data-config s3Uri="s3://tm-bedrock-output-data"
  --hyper-parameters batchSize="1",epochCount="5",learningRate="0.00001",learningRateWarmupSteps="5"
  --custom-model-kms-key-id arn:aws:kms:us-east-1:123456789012:key/1234abcd-1234-abcd-1234-abcd1234abcd
  --vpc-config 'subnetIds="subnet-0abcdabcdabcdabcd","subnet-01234123412341234",securityGroupIds="sg-01234abcd1234abcd"'

08 The command output should return the Amazon Resource Name (ARN) of the new model customization job:

{
	"jobArn": "arn:aws:bedrock:us-east-1:123456789012:model-customization-job/amazon.titan-text-lite-v1:0:4k/abcdabcdabcd"
}

09 Repeat steps no. 1 - 8 for each model customization job that you want to re-create, available in the selected AWS region.

10 Change the AWS cloud region by updating the --region command parameter value and repeat the Remediation process for other regions.