Retrieve secrets from Keeper Secrets Manager at Docker runtime
Dynamically retrieve secrets from the Keeper Vault when Docker containers execute
For a complete list of Keeper Secrets Manager features see the Overview
This page documents the Secrets Manager Docker Runtime integration. In order to utilize this integration, you will need:
Keeper Secrets Manager access (See the Quick Start Guide for more details)
Secrets Manager addon enabled for your Keeper account
Membership in a Role with the Secrets Manager enforcement policy enabled
A Keeper Secrets Manager Application with secrets shared to it
See the Quick Start Guide for instructions on creating an Application
The Keeper Secrets Manager (KSM) CLI Tool
See instructions on setting up the KSM CLI here
Keeper Secrets Manager integrates with the Docker Runtime so that you can dynamically retrieve a secret from the vault when the container executes.
The ksm
command is used to set environment variables when the container is started instead of hard-coding them into a deployment script. A real world example of this implementation is demonstrated below.
The official MySQL docker allows a user to set the MySQL root password and create a network accessible user via environment variables. The MySQL instance is then provisioned when a container is run.
The official MySQL dockerfile is below:
In the standard implementation, the ENTRYPOINT does the provisioning of the container and will use environmental variables that are passed in to set up MYSQL. The environmental variables referenced are the following:
MYSQL_ROOT_PASSWORD
MYSQL_USER
MYSQL_PASSWORD
MYSQL_DATABASE
The below steps will show how to initialize the MySQL database with secrets that are stored in the Keeper Vault.
Step 1: Create 2 Vault Records with Secrets
Create two records in the Vault that are managed by the Secrets Manager application. One record contains the root password. The other record contains the regular user, password and database values.
Make sure to copy the Record UID that appears in the vault records. These are used in Step 3 below when referencing the vault secrets.
Step 2: Create dockerfile that builds on the default MySQL dockerfile
We'll create a dockerfile that installs Keeper Secrets Manager CLI (ksm
) and then wraps the ENTRYPOINT with ksm exec
In the below dockerfile, the 4 environment variables are replaced using Keeper Notation. We are also passing in the Secrets Manager profile that points to the vault where the secrets are stored.
Step 3: Create a shell script to execute the docker build
To execute the docker build, the below script will pass in the Secrets Manager device configuration, root user Record UID and network user Record UID from the vault that contains the secrets.
The KSM CLI Docker includes a volume mount to both GLIBC (most Linux distributions) and MUSL (Alpine Linux) CLI binaries. The volume is /cli
. This directory can be mounted into another container using the volumes_from
in docker-compose or -v
from command line docker. The ksm executables exists in directory based on the version of C library your Linux distribution is using.
/cli/glibc/ksm
- For standard GLIBC distributions like Ubuntu, Debian, Fedora, and CentOS.
/cli/musl/ksm
- For Alpine Linux.
For example, the following is simple framework showing how to access the CLI binary.
The init
service will load the CLI docker. The container will start, display a CLI splash screen, and then will exit. Even though the container has stopped, the /cli
volume is still accessible from other containers.
The main
service will mount the CLI docker's volume under the directory /cli
using volumes_from
. The command
is overridden to run the GLIBC version of the KSM CLI. The command
is using the exec
function of the CLI. That will replace environment variables environment variable, that use the Keeper Notation, with a secret value. The exec
command, of the CLI, is running the printenv
application. That will print the environment variable, MY_LOGIN, that has been set to Keeper Notation, and has had its value replaced with a secret by the exec
command.
Similar to the examples above, the KSM CLI docker can be used to override the entrypoint and command for another vendor's Docker image without creating a custom Docker image.
This example, combines the first two examples.
For this example, it will be assumed the Docker image is being severed by the Docker Hub repository and the images' Dockerfile is stored on GitHub.
The first step is to determine what operating system distribution the vendor's Docker image is built upon. Often this can be determined by the tag name. For example, if the name has "alpine" in the image tag name, you'll know it's the Alpine Linux distribution.
If the image tag name does not indicate the distribution, then on the Docker Hub web page for the image, click on the tag name in the "Supported tags" section. This will display the content of the Dockerfile. The FROM statement will indicate the distribution the vendor has built their image upon. If it is not apparent from the FROM statement, you man check the Dockerfile of the FROM image due to inheritance.
MySQL 8.0.31 doesn't indicate the operating system distribution in the tag name. On the MySQL Docker Hub page, the 8.0.31 tag links to their GitHub repo. From the Dockerfile we can see the distribution is Oracle Linux.
The purpose of checking the distribution is to determine what version of the libc library is being used. Most distributions use GLIBC but some, mainly Alpine Linux, uses MUSL. This is needed to select the correct binary from the KSM CLI Docker image. If you select the wrong one, you will get an error like exec /cli/musl/ksm: no such file or directory
or exec /cli/glibc/ksm: no such file or directory
. For our example, Oracle Linux is a GLIBC distribution.
The next step is to determine the vendor's Docker image ENTRYPOINT and CMD. The Dockerfile will list the ENTRYPOINT and/or CMD.
From the MySQL Dockerfile, the ENTRYPOINT is ["docker-entrypoint.sh"]
and the CMD is ["mysqld"]
. This means the ENTRYPOINT will be prepended to the CMD, so when the container is started docker-entrypoint.sh mysqld
will be executed.
docker-compose.yml
The docker-compose uses two services.
The init service loads the keeper/keeper-secrets-manager-cli
Docker image volumes. This image image will start and exit, however the volumes will still be accessible after it exits.
The main service will run after the init service. This is done by using the docker-composes depends_on
directive. This service contains environment variables, with notation, that will be replaced by the KSM CLI exec
command and also includes the Base64 encoded configuration needed by the KSM CLI. The MYSQL_ environmental variables are used by the MySQL Docker image to provision the database.
The main services also will mount the volumes from the init service using the volumes_from
. The KSM CLI Docker image defined that volumes are exported, and where they are mounted in the main service container. The binaries are mounted in /cli
, followed by the libc version, and the ksm
binary name.
Since the MySQL image uses the Oracle distribution, a GLIBC distribution, the main service will use the /cli/glibc/ksm
binary.
The main service will override the ENTRYPOINT and CMD of the MySQL image. This is done using entrypoint and command. The entrypoint will use the KSM CLI exec
command to run the original ENTRYPOINT docker-entrypoint.sh
. The command is the same, however it needs to be set in the docker-compose.yml
else the service will just exit.
Based on the Docker image you are using, you may need override either ENTRYPOINT or CMD, or both.
When the services are brought up. The init service will run first and then exit with a code 0, which means it exited successfully execute. Then the main services will start up, execute the KSM CLI exec
command and run docker-entrypoint.sh
with mysqld
. At this point the environmental variable have been replaced with secrets, MySQL has been provisioned, and mysqld
is running.
If you have some great examples to contribute to this page, please ping us on Slack or email sm@keepersecurity.com.