Kubernetes (alternative)
Keeper Secrets Manager integration into Kubernetes for dynamic secrets retrieval
We recommend using the Kubernetes External Secrets Operator integration for most use cases. This document describes an alternate method of integration which does not utilize the External Secrets Operator.
- Retrieve secrets from the Keeper Vault within the Kubernetes
- Access secrets from the Keeper Vault in real-time across all pods
- Copy secure file attachments from the Keeper Vault to the local filesystem
This page documents the Secrets Manager Kubernetes integration. In order to utilize this integration, you will need:
- Secrets Manager addon enabled for your Keeper account
- Membership in a Role with the Secrets Manager enforcement policy enabled
Keeper Secrets Manager can be integrated into your K8s cluster for accessing Keeper secrets in real-time across all pods.
Using Commander, create a Secrets Manager device configuration for Kubernetes. Note that this configuration is not IP locked and it is pre-initialized.
Create the configuration in Commander using this command:
secrets-manager client add --app <APP NAME> --unlock-ip --config-init k8sExample:
1
My Vault> sm client add --app MyAdd --unlock-ip --config-init k8s
2
3
Successfully generated Client Device
4
====================================
5
6
Initialized Config:
7
8
apiVersion: v1
9
data:
10
config: ewog2N...ICIxMCIKfQ==
11
metadata:
12
name: ksm-config
13
namespace: default
14
type: Opaque
15
16
IP Lock: Disabled
17
Token Expires On: 2021-10-13 12:45:45
18
App Access Expires on: Never
In the example above, copy lines 8 through 14 and place them into a file called secret.yaml. Then, If you are using a machine with
kubectl installed and you have access to your cluster, add the KSM SDK config to your Kubernetes secrets.$ kubectl apply -f secret.yaml
For more information on creating Secrets Manger configurations, see the Configuration Documentation
Alternatively, you can create a configuration by generating a One Time Access Token with Commander (or the Vault UI) and then using the Keeper Secret Manager CLI to create a configuration as demonstrated below (replace XX:XXX) with the One Time Access Token.
$ ksm init k8s XX:XXX
apiVersion: v1
data:
config: ewog2N[...]ICIxMCIKfQ==
kind: Secret
metadata:
name: ksm-config
namespace: default
type: Opaque
If you are using a machine with
kubectl installed and you have access to your cluster, the parameter --apply can be set to automatically add the KSM SDK config to your Kubernetes secrets.The output of redeeming the token can be piped to a file and then applied via kubectl. For example:
$ ksm init k8s XX:XXX > secret.yaml
$ kubectl apply -f secret.yaml
secret/ksm-config created
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
spec:
containers:
- name: mycontainer
image: my_container:XXXXX
env:
- name: KSM_CONFIG
valueFrom:
secretKeyRef:
name: ksm-config
key: config
restartPolicy: Never
At runtime, the Keeper Developer SDKs running in the K8s cluster will use the environment variable KSM_CONFIG to retrieve the device configuration and communicate with the Keeper Vault.
Below is a simple example that will generate a deployment and service that displays database secrets via a web application. This example uses the Keeper Python Developer SDK for the web application. The SDK will get its configuration from a Kubernetes secret and then retrieve PostgreSQL database record information from the Keeper vault.
From the Keeper Vault, a "Database" record type is created using the following information.
Now let's create our web application. The web page can be created using any of the Developer SDKs. For this example, it will being using the Python SDK. A simple Flask application with one endpoint will display the HTML that contains the Vault record secrets. The secrets are retrieved using the Keeper Notation syntax.
from flask import Flask
from keeper_secrets_manager_core import SecretsManager
import os
app = Flask(__name__)
@app.route("/")
def hello_world():
sm = SecretsManager()
return """
<h1>Database</h1>
<ul>
<li>Type: {}</li>
<li>Host: {}</li>
<li>Port: {}</li>
<li>Login: {}</li>
<li>Password: {}</li>
</ul>
""".format(
sm.get_notation(os.environ.get("DB_TYPE")),
sm.get_notation(os.environ.get("DB_HOST")),
sm.get_notation(os.environ.get("DB_PORT")),
sm.get_notation(os.environ.get("DB_LOGIN")),
sm.get_notation(os.environ.get("DB_PASS")))
The next part is creating a Dockerfile. The Dockerfile below is based off of the Python Debian images from Docker Hub.
The Python SDK uses the cryptography module. This module requires the language Rust to be installed. Other Docker Hub images may provide Rust pre-installed.
FROM python:3.10.0-slim-bullseye
RUN apt-get update \
&& apt-get install -y gcc make libffi-dev curl libssl-dev \
&& apt-get clean
RUN pip3 install --upgrade pip wheel
# cryptology requires Rust to build
RUN curl https://sh.rustup.rs -sSf > /tmp/rust.sh \
&& chmod a+x /tmp/rust.sh \
&& /tmp/rust.sh -y
ENV PATH $PATH:/root/.cargo/bin
RUN pip3 install \
flask \
keeper-secrets-manager-core
RUN groupadd -g 5000 demo
RUN useradd -g demo demo
# Copy our application into the image
COPY demo.py /demo.py
USER demo
ENV FLASK_APP demo
EXPOSE 5000
CMD ["flask", "run"]
Next we will build a Docker image named ksm_demo.
$ docker build -t ksm_demo .
Assuming there is access to the Kubernetes cluster, the config can be generated from the One Time Access Token and automatically applied.
$ ksm init k8s --apply XX:XXXXXXXXXXX
secret/ksm-config created
Created secret for KSM config.
You can see the secret entry when you enter
kubectl get secret.$ kubectl get secret ksm-config
NAME TYPE DATA AGE
ksm-config Opaque 1 55s
You can then make a deployment and service for the ksm_demo Docker image. This example is going to name the file ksm_demo.yaml.
Defining which secrets you need, and the config for the SDK, happen in the env section in the list of containers. In this section the KSM_CONFIG environmental variable is defined to get it's value from the ksm-config Kubernetes secret, specifically the config key of the secret.
The other environmental variables are just list of name/values. The value is Keeper Notation which the web application will send to the notation retrieval method of the SDK.
The second record in the ksm_demo.yaml file is the Service definition. This can be changed to whatever works with your Kubernetes cluster. In our example, it uses an external IP address. It's safe to use one of your Kubernetes nodes IP address, 10.0.1.18 for the example.
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: ksm-demo-deployment
labels:
app: ksm-demo
spec:
replicas: 1
selector:
matchLabels:
app: ksm-demo
template:
metadata:
labels:
app: ksm-demo
spec:
nodeSelector:
kubernetes.io/hostname: work
dnsPolicy: "None"
dnsConfig:
nameservers:
- 10.0.1.207
- 1.1.1.1
containers:
- name: ksm-demo
image: ksm_demo:latest
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5000
protocol: TCP
env:
- name: KSM_CONFIG
valueFrom:
secretKeyRef:
name: ksm-config
key: config
- name: DB_TYPE
value: "keeper://IUCvqyWcx7sG-BGIK1R9-g/field/Type"
- name: DB_HOST
value: "keeper://IUCvqyWcx7sG-BGIK1R9-g/field/host[hostName]"
- name: DB_PORT
value: "keeper://IUCvqyWcx7sG-BGIK1R9-g/field/host[port]"
- name: DB_LOGIN
value: "keeper://IUCvqyWcx7sG-BGIK1R9-g/field/login"
- name: DB_PASS
value: "keeper://IUCvqyWcx7sG-BGIK1R9-g/field/password"
---
apiVersion: v1
kind: Service
metadata:
name: ksm-demo-service
spec:
ports:
- name: http
port: 5000
targetPort: 5000
protocol: TCP
selector:
app: ksm-demo
externalIPs:
- 10.0.1.18
At this point the deployment and service are ready to be applied.
$ kubectl apply -f ksm_demo.yaml
deployment/ksm-demo-deployment created
service/ksm-demo-service created
Wait until your deployment is ready. Either monitor it via the command line or Kubernetes Dashboard.
$ kubectl get deployment ksm-demo-deployment
NAME READY UP-TO-DATE AVAILABLE AGE
ksm-demo-deployment 1/1 1 1 46m
$ kubectl get svc ksm-demo-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ksm-demo-service ClusterIP 10.107.91.89 10.0.1.18 5000/TCP 56m
Finally, use a web browser and go to the external IP address, port 5000, and you will see the Keeper vault record database secrets.
Example Web Application displaying Keeper secrets
In this example a pod will be created that contains the stock NGINX docker image and SSL certificates retrieved from the Keeper Vault.
A Login record is created in the vault to hold the SSL certificate, private key, and certificate password.
A one time token is generated and added to the Kubernetes ConfigMap.
$ ksm init k8s XX:XXX > secret.yaml
$ kubectl apply -f secret.yaml
secret/ksm-config created
The example website is simply an index HTML page. The HTML can be stored in the ConfigMap and mounted in to the document root directory.
---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-html-config
namespace: default
labels:
app: nginx
data:
index.html: |
<html>
<head>
<title>Nginx Test Page</title>
</head>
<body>
<h1>Hello From Keeper Secrets Manager!</h1>
</body>
</html>
The
default.conf will be overwritten by our example. The certificate, key, and password will be placed in to the /etc/keys directory. For non-interactive startup, NGINX requires the certificate password be placed in a file and ssl_password_file to be included in the server configuration.---
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config
namespace: default
labels:
app: nginx
data:
default.conf: |
server {
listen 80 default;
server_name localhost;
location / {
root /var/www/nginx-default;
index index.html index.htm;
}
}
server {
listen 443 ssl;
server_name localhost;
ssl_certificate /etc/keys/example.com.crt;
ssl_certificate_key /etc/keys/example.com.key;
ssl_password_file /etc/keys/global.pass;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!MD5;
location / {
root /var/www/nginx-default;
index index.html index.htm;
}
}
The deployment for this example looks like the following:
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
labels:
app: nginx
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
initContainers:
- name: ksm
image: keeper/keeper-secrets-manager-writer:latest
env:
- name: KSM_CONFIG
valueFrom:
secretKeyRef:
name: ksm-config
key: config
- name: SECRETS
value: |
5x0v0VFwYj2VvuhamrJhzQ/field/password > file:/etc/keys/global.pass
5x0v0VFwYj2VvuhamrJhzQ/file/example.com.crt > file:/etc/keys/example.com.crt
5x0v0VFwYj2VvuhamrJhzQ/file/example.com.key > file:/etc/keys/example.com.key
volumeMounts:
- mountPath: "/etc/keys"
name: keys-volume
containers:
- name: nginx
image: nginx:1.21.4-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
protocol: TCP
- containerPort: 443
protocol: TCP
volumeMounts:
- mountPath: "/etc/keys"
name: keys-volume
- mountPath: "/etc/nginx/conf.d/default.conf"
name: nginx-config-file
subPath: default.conf
- mountPath: "/var/www/nginx-default/index.html"
name: nginx-html-file
subPath: index.html
volumes:
- name: keys-volume
emptyDir: {}
- name: nginx-config-file
configMap:
name: nginx-config
- name: nginx-html-file
configMap:
name: nginx-html-config
imagePullSecrets:
- name: my-docker-hub-secrets
The docker image
keeper/keeper-secrets-manager-writer is used for an initialization container. The container will retrieve the secrets and write them to disk so they can be used by NGINX. The secrets are written to the pod's emptyDir volume, mounted to /etc/keys. This directory will be removed when the pod is deleted.The main container will also mount the pod's emptyDir volume to /etc/keys. And will also mount the
default.conf into /etc/nginx/conf.d and the index.html file into the document root for the server.The last part is to create a service to access NGINX as shown below.
---
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
- name: https
port: 443
targetPort: 443
protocol: TCP
selector:
app: nginx
externalIPs:
- XXX.XXX.XXX.XX
The service can be tested by going to the external IP via https (ie https://XXX.XXX.XXX.XXX). Notice the lock in the address bar indicates the certificate is valid.
External Secrets is a Kubernetes operator that synchronizes secrets from External APIs and injects them into Kubernetes. For more information on how to setup External Secrets to synchronize secrets from your Keeper Vault into Kubernetes, visit the following:
At this point, you can now integrate Keeper Secrets Manager into your K8s deployments using any of the Secrets Manager SDKs.
Last modified 1mo ago