Keeper Secrets Manager integration into Kubernetes for dynamic secrets retrieval
We recommend using the Kubernetes External Secrets Operator integration for most use cases. This document describes an alternate method of integration which does not utilize the External Secrets Operator.
Retrieve secrets from the Keeper Vault within the Kubernetes
Access secrets from the Keeper Vault in real-time across all pods
Copy secure file attachments from the Keeper Vault to the local filesystem
For a complete list of Keeper Secrets Manager features see the Overview
This page documents the Secrets Manager Kubernetes integration. In order to utilize this integration, you will need:
Keeper Secrets Manager access (See the Quick Start Guide for more details)
Secrets Manager addon enabled for your Keeper account
Membership in a Role with the Secrets Manager enforcement policy enabled
A Keeper Secrets Manager Application with secrets shared to it
See the Quick Start Guide for instructions on creating an Application
Keeper Secrets Manager can be integrated into your K8s cluster for accessing Keeper secrets in real-time across all pods.
Using Commander, create a Secrets Manager device configuration for Kubernetes. Note that this configuration is not IP locked and it is pre-initialized.
Create the configuration in Commander using this command:
secrets-manager client add --app <APP NAME> --unlock-ip --config-init k8s
Example:
In the example above, copy lines 8 through 14 and place them into a file called secret.yaml. Then, If you are using a machine with kubectl
installed and you have access to your cluster, add the KSM SDK config to your Kubernetes secrets.
For more information on creating Secrets Manger configurations, see the Configuration Documentation
Alternatively, you can create a configuration by generating a One Time Access Token with Commander (or the Vault UI) and then using the Keeper Secret Manager CLI to create a configuration as demonstrated below (replace XX:XXX) with the One Time Access Token.
If you are using a machine with kubectl
installed and you have access to your cluster, the parameter --apply
can be set to automatically add the KSM SDK config to your Kubernetes secrets.
The output of redeeming the token can be piped to a file and then applied via kubectl. For example:
The KSM config can be pulled into your K8s containers using secrets.
At runtime, the Keeper Developer SDKs running in the K8s cluster will use the environment variable KSM_CONFIG to retrieve the device configuration and communicate with the Keeper Vault.
Below is a simple example that will generate a deployment and service that displays database secrets via a web application. This example uses the Keeper Python Developer SDK for the web application. The SDK will get its configuration from a Kubernetes secret and then retrieve PostgreSQL database record information from the Keeper vault.
From the Keeper Vault, a "Database" record type is created using the following information.
Now let's create our web application. The web page can be created using any of the Developer SDKs. For this example, it will being using the Python SDK. A simple Flask application with one endpoint will display the HTML that contains the Vault record secrets. The secrets are retrieved using the Keeper Notation syntax.
The next part is creating a Dockerfile. The Dockerfile below is based off of the Python Debian images from Docker Hub.
The Python SDK uses the cryptography module. This module requires the language Rust to be installed. Other Docker Hub images may provide Rust pre-installed.
Next we will build a Docker image named ksm_demo.
Assuming there is access to the Kubernetes cluster, the config can be generated from the One Time Access Token and automatically applied.
You can see the secret entry when you enter kubectl get secret
.
You can then make a deployment and service for the ksm_demo Docker image. This example is going to name the file ksm_demo.yaml.
Defining which secrets you need, and the config for the SDK, happen in the env section in the list of containers. In this section the KSM_CONFIG environmental variable is defined to get it's value from the ksm-config Kubernetes secret, specifically the config key of the secret.
The other environmental variables are just list of name/values. The value is Keeper Notation which the web application will send to the notation retrieval method of the SDK.
The second record in the ksm_demo.yaml file is the Service definition. This can be changed to whatever works with your Kubernetes cluster. In our example, it uses an external IP address. It's safe to use one of your Kubernetes nodes IP address, 10.0.1.18 for the example.
At this point the deployment and service are ready to be applied.
Wait until your deployment is ready. Either monitor it via the command line or Kubernetes Dashboard.
Finally, use a web browser and go to the external IP address, port 5000, and you will see the Keeper vault record database secrets.
In this example a pod will be created that contains the stock NGINX docker image and SSL certificates retrieved from the Keeper Vault.
A Login record is created in the vault to hold the SSL certificate, private key, and certificate password.
A one time token is generated and added to the Kubernetes ConfigMap.
The example website is simply an index HTML page. The HTML can be stored in the ConfigMap and mounted in to the document root directory.
The default.conf
will be overwritten by our example. The certificate, key, and password will be placed in to the /etc/keys
directory. For non-interactive startup, NGINX requires the certificate password be placed in a file and ssl_password_file
to be included in the server configuration.
The deployment for this example looks like the following:
The docker image keeper/keeper-secrets-manager-writer
is used for an initialization container. The container will retrieve the secrets and write them to disk so they can be used by NGINX. The secrets are written to the pod's emptyDir volume, mounted to /etc/keys. This directory will be removed when the pod is deleted.
Documentation for the Keeper Secrets Manager Writer can be found here.
The main container will also mount the pod's emptyDir volume to /etc/keys. And will also mount the default.conf
into /etc/nginx/conf.d and the index.html
file into the document root for the server.
The last part is to create a service to access NGINX as shown below.
The service can be tested by going to the external IP via https (ie https://XXX.XXX.XXX.XXX). Notice the lock in the address bar indicates the certificate is valid.
External Secrets is a Kubernetes operator that synchronizes secrets from External APIs and injects them into Kubernetes. For more information on how to setup External Secrets to synchronize secrets from your Keeper Vault into Kubernetes, visit the following:
Kubernetes External Secrets OperatorAt this point, you can now integrate Keeper Secrets Manager into your K8s deployments using any of the Secrets Manager SDKs.