AWS Elastic Container Service with KSM (Advanced)

Running Keeper Automator with the AWS ECS (Fargate) service and Keeper Secrets Manager for secret storage

Overview

This example demonstrates how to launch the Keeper Automator service in Amazon ECS with Fargate, while also demonstrating the use of Keeper Secrets Manager for retrieving the secret configuration for the published Docker container.

Keeper Setup

Since this deployment requires the use of Keeper Secrets Manager, this section reviews the steps needed to set up your Keeper vault and the SSL certificate for publishing the Automator service.

(1) SSL Certificate

Create an SSL Certificate as described from this page

When this step is completed, you will have two files: ssl-certificate.pfx and ssl-certificate-password.txt

(2) Create Shared Folder

Create a Shared Folder in your vault. This folder will not be shared to anyone except the secrets manager application.

(3) Add File Attachments

Create a record in the Shared Folder, and make note of the Record UID. Upload the SSL certificate and SSL certificate password files to a Keeper record in the shared folder.

(4) Add Automator Property File

Upload a new file called keeper.properties which contains the following content:

ssl_mode=certificate
ssl_certificate_file=/config/ssl-certificate.pfx
ssl_certificate_file_password=
ssl_certificate_key_password=
automator_host=localhost
automator_port=443
disable_sni_check=true
persist_state=true

The notable line here is the disable_sni_check=true which is necessary when running the Automator service under a managed load balancer.

Your shared folder and record should look something like this:

Shared Folder containing a record with 3 files

(5) Create KSM Application

Create a Keeper Secrets Manager ("KSM") application in your vault. If you are not familiar with secrets manager, follow this guide. The name of this application is "Automator" but the name does not matter.

KSM Application

(6) Attach the KSM application to the shared folder

Edit the Shared Folder and add the Automator application to this folder.

Assign Application to Shared Folder

(7) Create a KSM Device Configuration

Open the secrets manager application, click on "Devices" tab and click "Add Device". Select a base64 configuration. Download and save this configuration for use in the ECS task definition.

Create base64 configuration

AWS Setup

(1) Create a VPC

If your VPC does not exist, a basic VPC with multiple subnets, a route table and internet gateway must be set up. In this example, there are 3 subnets in the VPC with an internet gateway as seen in the resource map below:

VPC Setup

(2) Create CloudWatch Log Group

Go to CloudWatch > Create log group

Name the log group "automator-logs".

Create CloudWatch Log Group

(3) Create an Execution IAM Role

Go to IAM > Create role

Select AWS service

Then search for Elastic Container Service and select it.

Select "Elastic Container Service Task" and click Next

Add the "AmazonECSTaskExecution" policy to the role and click Next

Assign the name "ECSTaskWritetoLogs" and then create the role.

Make note of the ARN for this Role, as it will be used in the next steps.

In this example, it is arn:aws:iam::373699066757:role/ECSTaskWritetoLogs

Make note of ARN

(4) Create a Security Group for ECS

Go to EC2 > Security Groups and click on "Create security group"

Depending on what region your Keeper tenant is hosted, you need to create inbound rules that allow https port 443. The list of IPs for each tenant location is on this page. In the example below, this is the US data center.

• We also recommend adding your workstation's external IP address for testing and troubleshooting.

Assign a name like "MyAutomatorService" and then click "Create".

Create ECS Security Group

After saving the security group, edit the inbound rules again. This time, add HTTPS port 443 and select the security group in the drop-down. This will allow the load balancer to monitor health status and distribute traffic.

(5) Create Security Group for EFS

We'll create another security group that controls NFS access to EFS from the cluster.

Go to EC2 > Security Groups and click on "Create security group"

Set a name such as "MyAutomatorEFS".

Select Type of "NFS" and then select Custom and then the security group that was created in the prior step for the ECS cluster. Click "Create security group".

Note the security group ID for the next step. In this case, it is sgr-089fea5e4738f3898

(6) Create two Elastic File System volumes

At present, the Automator service needs access to two different folders. In this example setup, we are creating one volume to store the SSL certificate and SSL passphrase files. The second volume stores the property file for the Automator service. These 3 files are in your Keeper record.

Go to AWS > Elastic File System and click Create file system

Call it "automator_config" and click Create

Again, go to Elastic File System and click Create file system. Call this one automator_settings and click Create.

Note the File system IDs displayed. These IDs (e.g. fs-xxx) will be used in the ECS task definition.

After a minute, the 2 filesystems will be available. Click on each one and then select the "Network" tab then click on "Manage".

Manage the network security group on EFS

Change the security group for each subnet to the one created in the above step (e.g. "MyAutomatorEFS") and click Save. Make this network change to both filesystems that were created.

Change the security group on EFS

(7) Create Elastic Container Service Cluster

Navigate to the Amazon Elastic Container Service.

Select "Create cluster" and assign the cluster name and VPC. In this example we are using the default "AWS Fargate (serverless)" infrastructure option.

• The Default namespace can be called "automator"

• The "Infrastructure" is set to AWS Fargate (serverless)

• Click Create

Create ECS Cluster with AWS Fargate

(8) Create ECS Task Definition

In your favorite text editor, copy the below JSON task definition file and save it.

Make the following changes to the JSON file:

• Change the XXXCONFIGXXX value to a base64 config from Keeper Secrets Manager created in the beginning of this guide

• Change the 3 instances of "XXXXX" to the Record UID containing the SSL certificate, SSL certificate password and settings file in your vault shared folder which KSM is accessing.

• Change the two File system IDs (fs-XXX) to yours from the above steps

• Change the XXX for the role ID as specific to your AWS role

• Change the eu-west-1 value in two spots to the region of your ECS service

{
    "family": "automator",
    "containerDefinitions": [
        {
            "name": "init",
            "image": "keeper/keeper-secrets-manager-writer:latest",
            "cpu": 1024,
            "memory": 1024,
            "portMappings": [],
            "essential": false,
            "environment": [
                {
                    "name": "KSM_CONFIG",
                    "value": "XXXCONFIGXXX"
                },
                {
                    "name": "SECRETS",
                    "value": "XXXXX/file/ssl-certificate.pfx > file:/usr/mybin/config/ssl-certificate.pfx\nXXXXX/file/ssl-certificate-password.txt > file:/usr/mybin/config/ssl-certificate-password.txt\nXXXXX/file/keeper.properties > file:/usr/mybin/settings/keeper.properties"
                }
            ],
            "mountPoints": [
                {
                    "sourceVolume": "automatorconfig",
                    "containerPath": "/usr/mybin/config"
                },
                {
                    "sourceVolume": "automatorsettings",
                    "containerPath": "/usr/mybin/settings"
                }
            ],
            "volumesFrom": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "automator-logs",
                    "awslogs-region": "eu-west-1",
                    "awslogs-stream-prefix": "container-1"
                }
            }
        },
        {
            "name": "main",
            "image": "keeper/automator:latest",
            "cpu": 1024,
            "memory": 4096,
            "portMappings": [
                {
                    "containerPort": 443,
                    "hostPort": 443,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "environment": [],
            "mountPoints": [
                {
                    "sourceVolume": "automatorconfig",
                    "containerPath": "/usr/mybin/config"
                },
                {
                    "sourceVolume": "automatorsettings",
                    "containerPath": "/usr/mybin/settings"
                }
            ],
            "volumesFrom": [],
            "dependsOn": [
                {
                    "containerName": "init",
                    "condition": "SUCCESS"
                }
            ],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "automator-logs",
                    "awslogs-region": "eu-west-1",
                    "awslogs-stream-prefix": "container-2"
                }
            }
        }
    ],
    "executionRoleArn": "arn:aws:iam::XXX:role/ECSTaskWritetoLogs",
    "networkMode": "awsvpc",
    "volumes": [
        {
            "name": "automatorconfig",
            "efsVolumeConfiguration": {
                "fileSystemId": "fs-XXX",
                "rootDirectory": "/",
                "transitEncryption": "ENABLED"
            }
        },
        {
            "name": "automatorsettings",
            "efsVolumeConfiguration": {
                "fileSystemId": "fs-XXX",
                "rootDirectory": "/",
                "transitEncryption": "ENABLED"
            }
        }
    ],
    "requiresCompatibilities": [
        "FARGATE"
    ],
    "cpu": "2048",
    "memory": "5120"
}

Next, go to Elastic Container Service > Task definitions > Create Task from JSON

Create task definition with JSON

Remove the existing JSON and copy-paste the contents of the JSON file above into the box, then click Create.

Task definition

This task definition can be modified according to your instance CPU/Memory requirements.

(9) Upload SSL Cert to AWS Certificate Manager

In order for an application load balancer in AWS to serve requests for Automator, the SSL certificate must be managed by the AWS Certificate Manager.

Go to AWS Certificate Manager and Click on Import

On your workstation, we need to convert the SSL certificate (.pfx) file to a PEM-encoded certificate body, PEM-encoded private key and PEM-encoded certificate chain.

Since you already have the .pfx file, this can be done using the below openssl commands:

Download the ssl-certificate.pfx file and the certificate password locally to your workstation and enter the below commands:

• Generate the PEM-encoded certificate body

openssl pkcs12 -in ssl-certificate.pfx -out automator-certificate.pem -nodes
openssl x509 -in automator-certificate.pem -out certificate_body.crt

• Generate the PEM-encoded private key

openssl pkey -in automator-certificate.pem -out private_key.key

• Generate the PEM-encoded certificate chain

openssl crl2pkcs7 -nocrl -certfile automator-certificate.pem | openssl pkcs7 -print_certs -out certificate_chain.crt

Copy the contents of the 3 files into the screen, e.g.

cat certificate_body.crt | pbcopy
   (paste into Certificate body section)
   
cat private_key.key | pbcopy
   (paste into Certificate private key section)
   
cat certificate_chain.crt | pbcopy
   (paste into Certificate chain section)

Import Certificate for Automator Service

(10) Create a Target Group

Go to EC2 > Target Groups and click Create target group

• Select "IP Addresses" as the target type

• Enter the Target group name of "automatortargetgroup" or whatever you prefer

• Select HTTPS Protocol with Port 443

• Select the VPC which contains the ECS cluster

• Select HTTP1

• Under Health checks, select the Health check protocol "HTTPS"

• Type /health as the Health check path

• Click Next

• Don't select any targets yet, just click Create target group

(11) Create Application Load Balancer (ALB)

Go to EC2 > Load balancers > Create load balancer

Select Application Load Balancer > Create

• Assign name such as "automatornalb" or whatever you prefer

• Scheme is "Internet-facing"

• IP address type: IPv4

• In the Network Mapping section, select the VPC and the subnets which will host the ECS service.

• In the Security groups, select "MyAutomatorService" as created in Step 4.

• In the Listeners and routing section, select HTTPS port 443 and in the target group select the Target group as created in the prior step (automatortargetgroup).

• In the Secure listener settings, select the SSL certificate "from ACM" that was uploaded to the AWS Certificate Manager in Step 9.

• Click Create load balancer

(12) Create ECS Service

Go to Elastic Container Service > Task definitions > Select the task created in Step 8.

From this Task definition, click on Deploy > Create Service

Create service

• Select Existing cluster of "automator"

• Assign Service name of "automatorservice" or whatever name you prefer

• Important: For the number of Desired tasks, set this to 1 for right now. After configuration, we will increase to the number of tasks you would like to have running.

• Under Networking, select the VPC, subnets and replace the existing security group with the ECS security group created in Step 4. In this case, it is called "MyAutomatorService".

• For Public IP, turn this ON.

• Under Load balancing, select Load balancer type "Application Load Balancer"

• Use an existing load balancer and select "automatoralb" created in Step 11.

• Use an existing listener, and select the 443:HTTPS listener

• Use an existing target group, and select the Target Group from Step 10

• Click Create

Environment Section of ECS Service

Deployment configuration

Networking section of ECS Service

After a few minutes, the service should start up.

(13) Update DNS

Assuming that the DNS name is hosted and managed by Route53:

Go to Route53 > Create or Edit record

• Create an A-record

• Set as "Alias"

• Route traffic to "Alias to Application and Classic Load Balancer"

• Select AWS Region

• Select the "automatoralb" Application Load Balancer

• Select "Simple Routing"

• Select "Save"

The next step is to configure Automator using Keeper Commander while only having one task running.

(14) Install Keeper Commander

At this point, the service is running but it is not able to communicate with Keeper yet.

On your workstation, server or any computer, install the Keeper Commander CLI. This is just used for initial setup. The installation instructions including binary installers are here: Installing Keeper Commander After Commander is installed, you can type keeper shell to open the session, then login using the login command. In order to set up Automator, you must login as a Keeper Administrator, or an Admin with the ability to manage the SSO node.

$ keeper shell

My Vault> login [e-mail]
.
.
My Vault>

(15) Initialize with Commander

Login to Keeper Commander and activate the Automator using a series of commands, starting with automator create

My Vault> automator create --name="My Automator" --node="Azure Cloud"

The Node Name (in this case "Azure Cloud") comes from the Admin Console UI as seen below.

Automator Create

The output of the command will display the Automator settings, including metadata from the identity provider.

                    Automator ID: 1477468749950
                            Name: My Automator
                             URL: 
                         Enabled: No
                     Initialized: No
                          Skills: Device Approval

Run the "automator edit" command as displayed below, which sets the URL and also sets up the skills (team, team_for_user and device).

automator edit --url https://<application URL> --skill=team --skill=team_for_user --skill=device "My Automator"

Next we exchange keys: The enterprise private key encrypted with the Automator public key is provided to Automator:

automator setup "My Automator"

Initialize the Automator with the new configuration

automator init "My Automator"

Enable the service

automator enable "My Automator"

At this point, the configuration is complete.

For automated health checks, you can use the below URL:

https://<server>/health

Example curl command:

$ curl https://automator.lurey.com/health
OK

In this example setup, the load balancer will be sending health checks to the target instances.

(16) Testing the User Experience

Now that Keeper Automator is deployed with a single task running, you can test the end-user experience. No prompts for approval will be required after the user authenticates with the SSO identity provider.

The easiest way to test is to open an incognito mode window to the Keeper Web Vault and login with SSO Cloud. You will not be prompted for device approval.

Vault Login

SSO Login

Automated Approval

Vault Decrypted

Assuming that the approval worked, you can now increase the number of tasks running.

(17) Update Task Definition

From the ECS screen, open the automator service and click "Update Service". Then set the number of tasks that you would like to have running.

Update service

Set the desired tasks

Logging and Monitoring

The Automator logs can be searched and monitored in the "Logs" tab of the ECS service, or in CloudWatch.

Logging and Monitoring