AWS Secrets Manager Storage

Store Secrets Manager connection details in AWS Secrets Manager. No local file required.

Keeper Secrets Manager integrates with AWS Secrets Manager so the KSM client configuration can be stored as a secret inside your AWS account instead of as a local file on the host. The configuration is fetched on each SDK startup, with no plaintext or encrypted file kept on disk.

Looking for AWS KMS encryption of a local config file? See AWS KMS Encryption. Thatintegration keeps the config on disk and wraps it with an AWS KMS customer master key. The two AWS integrations solve different problems and use different AWS services; pick the one that fits your deployment shape.

Features

• Store the KSM client configuration as a secret in AWS Secrets Manager, with no local file required
• Fetch the configuration over IAM-authenticated AWS APIs on every SDK startup
• Suitable for ephemeral compute (Lambda, ECS tasks, autoscaling EC2) where baking a config file into the image is undesirable
• AWS Secrets Manager handles encryption at rest internally with its own KMS keys; no client-side key management required
• Works with all KSM Python SDK functionality

Prerequisites

• Supports the Python Secrets Manager SDK
• Requires Python 3.9.2+ (pin to keeper-secrets-manager-storage<1.1.0 for Python 3.6–3.8)
• Requires the boto3 package
• AWS account with a pre-created AWS Secrets Manager secret and IAM permissions to read and write it

Setup

1. Install Module

pip3 install keeper-secrets-manager-storage
pip3 install boto3

2. Configure AWS Access

The Python SDK reads and writes the KSM configuration as a single AWS Secrets Manager secret. You'll need:

• A secret name (or full ARN) that the SDK will read from and write to. The default name is ksm-config if none is provided.
• AWS credentials with permission to read and write the secret.

The IAM permissions on the target secret are:

• secretsmanager:GetSecretValue
• secretsmanager:PutSecretValue

Example AWS policy granting access to a single secret:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "KsmConfigSecretAccess",
            "Effect": "Allow",
            "Action": [
                "secretsmanager:GetSecretValue",
                "secretsmanager:PutSecretValue"
            ],
            "Resource": "arn:aws:secretsmanager:<region-id>:<account-id>:secret:<unique-secret-name>"
        }
    ]
}

The secret must be pre-created. AwsSecretStorage reads the secret at construction time and fails if the secret does not exist. Before running your application for the first time, create the secret with at least an empty JSON object ({}) as its value:

aws secretsmanager create-secret --name ksm-config --secret-string '{}'

You can also create the secret via the AWS Console or your IaC tool. The SDK does not create the secret on first run.

The boto3 package looks for AWS credentials in the standard locations: instance role, ~/.aws/credentials, environment variables (AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_DEFAULT_REGION), or a named profile. See the AWS documentation for setup.

3. Add AWS Secrets Manager Storage to Your Code

Use AwsSecretStorage as your Secrets Manager storage in the SecretsManager constructor. The class supports four credential resolution modes; pick the one that matches your deployment.

Upgrading from v1.0.x? v1.1.0 includes these behavioral changes affecting AwsSecretStorage:

• AwsSecretStorage.__init__ now eagerly loads the config on construction (matching every other backend). Invalid credentials or missing secrets surface as exceptions immediately instead of on the first SDK call.
• AwsSecretStorage now accepts any valid JSON dict from the secret. It no longer requires the secret value to contain a privateKey field.
• AwsSecretStorage.__load_config() now raises when the underlying AWS Secrets Manager call fails. In v1.0.x, errors were logged but not propagated, leaving config = {} with no signal.
• AwsConfigProvider._get_instance_region and read_config now raise on failure instead of silently returning empty values.
• AwsSecretStorage is now thread-safe for concurrent reads and writes.

Using EC2 instance role (default)

The default constructor uses the EC2 instance's IAM role with a fallback to the default AWS profile if the instance role lookup fails. Region is auto-detected from the EC2 Instance Metadata Service (IMDS).

from keeper_secrets_manager_core import SecretsManager
from keeper_secrets_manager_storage.storage_aws_secret import AwsSecretStorage

# aws_key is the secret name, secret ID, or full ARN
config = AwsSecretStorage(aws_key="ksm-config")

secrets_manager = SecretsManager(config=config, verify_ssl_certs=True)
all_records = secrets_manager.get_secrets()

This mode requires either an EC2 instance or a full ARN as aws_key. Region auto-detection uses IMDS, which is only available on EC2. If you pass a short secret name (e.g. "ksm-config") outside EC2, construction raises "Failed to determine AWS region for EC2 instance." On a workstation or other non-EC2 environment, either pass a full ARN (the region is parsed from it) or use one of the other credential modes below.

Using the default AWS profile

For code running outside EC2 (a workstation, a CI runner, a container with mounted AWS credentials):

from keeper_secrets_manager_core import SecretsManager
from keeper_secrets_manager_storage.storage_aws_secret import AwsSecretStorage

config = AwsSecretStorage()
config.from_default_config(aws_key="ksm-config")

secrets_manager = SecretsManager(config=config, verify_ssl_certs=True)
all_records = secrets_manager.get_secrets()

Using a named AWS profile

from keeper_secrets_manager_core import SecretsManager
from keeper_secrets_manager_storage.storage_aws_secret import AwsSecretStorage

config = AwsSecretStorage()
config.from_profile_config(aws_key="ksm-config", profile="ksm-prod")

secrets_manager = SecretsManager(config=config, verify_ssl_certs=True)
all_records = secrets_manager.get_secrets()

Using explicit credentials

from keeper_secrets_manager_core import SecretsManager
from keeper_secrets_manager_storage.storage_aws_secret import AwsSecretStorage

config = AwsSecretStorage()
config.from_custom_config(
    aws_key="arn:aws:secretsmanager:us-east-2:123456789012:secret:ksm-config-aBcDeF",
    aws_access_key_id="AK[...]FIF",
    aws_secret_access_key="/[...]/g3",
    region="us-east-2",
)

secrets_manager = SecretsManager(config=config, verify_ssl_certs=True)
all_records = secrets_manager.get_secrets()

The aws_key parameter accepts any of:

• A secret name: ksm-config
• A secret ID: ksm-config-aBcDeF
• A full ARN: arn:aws:secretsmanager:us-east-2:123456789012:secret:ksm-config-aBcDeF

When aws_key is empty, the SDK reads from the KSM_AWS_SECRET environment variable. If that is also unset, the default secret name ksm-config is used.

Using AWS Secrets Manager Storage

Once setup, AwsSecretStorage supports all Secrets Manager SDK functionality. Each SDK startup performs at least one GetSecretValue call to fetch the configuration; writes (such as the one-time initialization with a one-time access token) issue a PutSecretValue.

Behavior on first-time initialization

AwsSecretStorage reads the secret value on every construction, where __init__ calls __load_config, which calls provider.read_config()). If the secret does not exist, or its value is not valid JSON, construction fails with a ValueError. Pre-create the secret with {} before first use; see the warning in the IAM section above.

After construction, the first one-time-token-driven PutSecretValue writes the populated configuration into the existing secret. Subsequent SDK runs read that configuration and skip re-initialization.

Removing the configuration

Calling config.delete_all() clears the local in-memory configuration and writes an empty JSON object back to the AWS secret. The secret itself is not deleted from AWS Secrets Manager, only its value is cleared. To remove the secret entirely, use the AWS Console, AWS CLI, or your IaC tool.

You're all set and ready to use Secrets Manager with AWS Secrets Manager storage. 👍

Check out the KSM SDKs documentation for more examples and functionality