Automating with AWS Lambda

Run Commander in the AWS Cloud

About

Commander is a powerful tool that can solve many issues and provide valuable information about your Keeper Security environment. In addition to using Commander on a local desktop or server, Commander can be run in a cloud environment such as AWS for performing scheduled or on-demand requirements.

In this example, we will demonstrate how to use Commander with AWS Lambda to run user and usage reports on a scheduled basis.

Prerequisites

• Commander

  • See the Installation and Setup page for download details

• A Keeper user account with a Master Password login method (SSO login and MFA will not work without human interaction)

• Access to AWS with permissions to create Lambda functions and associated layers, along with access to AWS CloudShell

Steps

Create a Lambda Layer With AWS CloudShell

Setup

Commander needs to be packaged on a machine that matches what it will run on in AWS Lambda. To do this, we can create the Commander package in CloudShell.

AWS CloudShell comes with Python 3.7 pre-installed, but just to be sure you are using a supported version of Python (versions 3.6 - 3.11) to build your Lambda Layer, go ahead and check the version of the installed Python interpreter in your CloudShell console.

$ python3 --version
Python 3.9.16

In case the Python version you're running is not supported by Commander, install one of the supported versions listed above prior to proceeding.

Building the Layer Content

For the next part of the process, we provide a convenient shell script for you to run from within your CloudShell environment, which will create the zip file that contains the keepercommander package needed for our Lambda Layer.

This script is intended to simplify and streamline much of the layer-content packaging process, both by encapsulating the various command calls that are standard for the process, and by abstracting away some build-process quirks specific to the keepercommander package and its dependencies. As such, we highly recommend using this approach over a more generic one, as it is much less likely to be error-prone.

View Script

package_layer_content.sh

#!/usr/bin/env bash

#   To create a `keepercommander` dependency layer for your AWS Lambda function :
#   1. Upload this script to any folder in your CloudShell environment.
#   2. (Optional) Upload your project's `requirements.txt` file  to the same folder.
#   3. In that folder, run
#             source ./package_layer_content.sh
#   4. There should now be a file named `commander-layer.zip` that can be uploaded
#     to your S3 bucket, where it can then be used to create a new Lambda layer

MAX_LIB_SIZE=262144000
LAYER_FILENAME='commander-layer.zip'
LAYER_PATH=$(pwd)/$LAYER_FILENAME
LIB_DIR='python'
VENV='commander-venv'
OTHER_DEPS='requirements.txt'

# Clean up previous artifacts
test -f $LAYER_FILENAME && rm $LAYER_FILENAME
test -d $LIB_DIR && rm -rf $LIB_DIR
test -d $VENV && rm -rf $VENV

# Create package folder to zip
mkdir $LIB_DIR

# Create and run virtual environment
python -m venv $VENV
source ./$VENV/bin/activate

# Install dependencies and package
pip install cryptography --platform manylinux2014_x86_64 --only-binary=:all: -t $LIB_DIR
pip install keepercommander -t $LIB_DIR

if test -f $OTHER_DEPS; then
  pip install -r $OTHER_DEPS -t $LIB_DIR
fi

deactivate

# Check uncompressed library size
LIB_SIZE=$(du -sb $LIB_DIR | cut -f 1)
LIB_SIZE_MB=$(du -sm $LIB_DIR | cut -f 1)

if [ "$LIB_SIZE" -ge $MAX_LIB_SIZE ]; then
  echo "*****************************************************************************************************************"
  echo 'Operation was aborted'
  echo "The resulting layer has too many dependencies and its size ($LIB_SIZE_MB MB) exceeds the maximum allowed (~262 MB)."
  echo 'Try breaking up your dependencies into smaller groups and package them as separate layers.'
  echo "*****************************************************************************************************************"
else
  zip -r $LAYER_FILENAME $LIB_DIR
  echo "***************************************************************************"
  echo "***************************************************************************"
  echo 'Lambda layer file has been created'
  printf "To download, copy the following file path: %s\n%s\n$LAYER_PATH%s\n%s\n"
  echo 'and click on "Actions" in the upper-right corner of your CloudShell console'
  echo "***************************************************************************"
fi

# Clean-up
rm -rf $LIB_DIR
rm -rf $VENV

2KB

package_layer_content.sh

Bash script for packaging keepercommander Lambda Layer content

To use the script provided above, perform the following steps after downloading the file:

1. Upload the script to any folder (preferably an empty one) in your CloudShell environment

   • (Optional) If your project has a requirements.txt file containing a list of its dependencies, you can upload it to the same folder to include those dependencies in the resulting layer in addition to the keepercommander package.

2. In that same folder, run the following command in the terminal:

source ./package_layer_content.sh

3. You should now have a zip file (commander-layer.zip) in your current folder, which represents the content of your Lambda Layer.

There is a size limit on packaged Lambda layer content (even if stored in S3). So if you try to include additional dependencies by providing the requirements.txt file for the script above, and it results in the total content size exceeding this limit, the resulting zip file will be unusable. Hence, no layer content will be output when the script detects this scenario, and a helpful message will be shown to the user instead.

A relatively simple solution is to break up your dependencies into smaller groups to be packaged into corresponding separate layers. You can remove some (or all) dependencies from requirements.txt and run the script again. Any dependencies excluded from the resulting package can then be packaged separately into another layer using the standard packaging process.

Creating / Updating a Layer from the Content Zip File

Because the resulting zip file is going to be bigger than 50MB (the maximum allowed to be uploaded directly to a Lambda Layer), we'll have to first upload it to an AWS S3 Bucket, and then link the resulting S3 item to our Lambda Layer.

There are multiple ways to complete the remaining steps just mentioned, and if you prefer a GUI-based route, using the AWS Console is a perfectly valid option at this point. But since we're already in our CloudShell environment, using its built-in AWS CLI command-line tool seems like the simplest and most direct way forward, so that is the method we'll show here.

1. First, we need to upload our zip file to AWS S3. If you didn't previously create a S3 bucket for this task, you can do so by running the following command in CloudShell:

$ aws s3 mb <bucket-name>

where <bucket-name> is required to be a globally unique name, so you may have to try at least a few times until one is found.

2. Upload the newly-packaged zip file from CloudShell to your S3 bucket

$ aws s3 cp ./commander-layer.zip 's3://<bucket-name>'

3. Publish the Lambda layer with the uploaded content

$ aws lambda publish-layer-version --layer-name <layer-name> \
--description <layer-description> \
--content "S3Bucket=<bucket-name>,S3Key=commander-layer.zip" \
--compatible-runtimes python<your-version>

Create a Lambda

In AWS Lambda, use the Lambda editor to write a Python function.

The lambda_handler function is triggered by Lambda when it is processed.

See a complete example of a Commander Lambda function below:

#  _  __
# | |/ /___ ___ _ __  ___ _ _ ®
# | ' </ -_) -_) '_ \/ -_) '_|
# |_|\_\___\___| .__/\___|_|
#              |_|
#
# Keeper Commander
# Copyright 2024 Keeper Security Inc.
# Contact: [e-mail]
#
#
import json
import os
import datetime
from typing import Optional

from email.mime.application import MIMEApplication
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText

import boto3

from keepercommander import api
from keepercommander.commands.enterprise import UserReportCommand
from keepercommander.commands.security_audit import SecurityAuditReportCommand
from keepercommander.params import KeeperParams


# Create report (format: JSON) combining data from 2 existing Commander reports
def create_user_report(params):  # type: (KeeperParams) -> Optional[str]
    user_report = UserReportCommand()
    user_report_data = user_report.execute(params, format='json')
    data = json.loads(user_report_data)
    users = {x['email']: x for x in data}
    security_audit_report = SecurityAuditReportCommand()
    security_audit_report_data = security_audit_report.execute(params, format='json')
    if security_audit_report_data:
        data = json.loads(security_audit_report_data)
        for x in data:
            if 'email' in x:
                email = x['email']
                if email in users:
                    user = users[email]
                    for key in x:
                        if key not in user:
                            if key not in ('node_path', 'username'):
                                user[key] = x[key]
                else:
                    users[email] = x

    return json.dumps(list(users.values()), indent=2)


# This Lambda's entry point
def lambda_handler(event, context):
    params = get_params()
    api.login(params)
    api.query_enterprise(params, True)
    
    # Log Commander-related issues (e.g., incorrect credentials) 
    # using AWS's built-in logging module and abort
    if not params.session_token:
        print('Not connected') 
        return 'Error: See Lambda log for details'
    if not params.enterprise:
        print('Not enterprise administrator')
        return 'Error: See Lambda log for details'
    
    # Generate and send report 
    report = create_user_report(params)
    response = email_result(report)
    return response


# Email report data (as JSON attachment) to recipient specified in this Lambda
# function's environment variables
def email_result(report):
    sender = os.environ.get('KEEPER_SENDER')
    sendto = os.environ.get('KEEPER_SENDTO')
    region = 'us-east-1'
    ses_client = boto3.client('ses', region_name=region)

    message = MIMEMultipart('mixed')
    message['Subject'] = 'Keeper Commander User Security Report With  CSV (attached)'
    message['To'] = sendto
    message['From'] = sender
    now = datetime.datetime.now()

    body = MIMEText(f'User Report Output created and sent at {now}', 'plain')
    message.attach(body)

    attachment = MIMEApplication(report)
    attachment.add_header(
        'Content-Disposition',
        'attachment',
        filename='user-report.json'
    )
    message.attach(attachment)

    response = ses_client.send_raw_email(
        Source=message['From'],
        Destinations=[sendto],
        RawMessage={'Data': message.as_string()}
    )
    
    return response


# Get required Commander parameters from environment variables
def get_params():
    user = os.environ.get('KEEPER_USER')
    pw = os.environ.get('KEEPER_PASSWORD')
    server = os.environ.get('KEEPER_SERVER')
    private_key = os.environ.get('KEEPER_PRIVATE_KEY')
    token = os.environ.get('KEEPER_DEVICE_TOKEN')
    my_params = KeeperParams()
    my_params.user = user
    my_params.password = pw
    my_params.server = server
    my_params.device_private_key = private_key
    my_params.device_token = token
    return my_params

The function is made up of several parts:

The lambda_handler function

This function is called when the lambda is triggered, all other functions should be called from this one.

Gathering Commander Parameters

Commander uses some parameters in order to authenticate the user account. We will attach these as environment variables and inject them into the Lambda. See further instructions below.

Commander Functions

The above steps are all that are required to run Commander in Lambda, once they have been done and Commander SDK code can be performed.

In this example, we run a user status report and send the results to an email address.

Setting Commander Parameters

Commander uses some parameters to authenticate the user account and identify what Keeper region to access. In order to pass these parameters into Lambda, we will set them as environment variables.

Gathering parameters

Commander automatically creates the required parameters when you login to the CLI. The easiest way to generate the required parameters is to login to the Commander CLI on your machine.

To get the Commander parameters, open the generated config.json file. By default this is located in the Users/[your username]/.keeper/ folder on your machine. See the config file documentation for more information.

You should see a file that looks similar to this:

{
    "clone_code": "36[...]A0g",
    "user": "[e-mail]",
    "server": "keepersecurity.com",
    "private_key": "sxv[...]oz3p=fzw",
    "device_token": "xko[...]r2IxdiQ"
}

Setting Parameters in AWS Lambda

To set the required Commander parameters as environment variables, first head to the lambda configuration and select "Environment Variables".

Set each commander parameter as an environment variable to be used by the Lambda.

You will also need to add your keeper master password to be used to login to Commander.

Environment Variable Name	Value	Example
KEEPER_USER	Keeper user account email address (config user field)	[e-mail]
KEEPER_SERVER	Keeper server domain (config server field)	keepersecurity.com
KEEPER_CLONE_CODE	clone_code field from config	36df3[...]A0dsa4g
KEEPER_PRIVATE_KEY	private_key field from config	sxv[...]oz3p=fzw
KEEPER_DEV_TOKEN	device_token field from config	xko[...]r2IxdiQ
KEEPER_PASSWORD	Password for keeper account	*****
KEEPER_SENDER	Email address to send emails from	[e-mail]
KEEPER_SENDTO	Email address to send emails to	[e-mail]

We are using environment variables to set the email addresses to send from and to for this example code. If you are not sending an email in your script, these are not needed

Configure Lambda

Set Timeout

In the general configuration section of the Lambda configuration, it is recommended to change the timeout value. Some Commander functions take time, so if your script takes longer to complete than this number, the Lambda will automatically end without finishing.

You can set this to any number you are comfortable with. A value of 300 seconds (5 minutes) should be more than enough time for most processes.

Select Layer

In the Lambda editor, select the layer that you created above to include the Commander package in your Lambda build.

Create run Schedule

Create an EventCloud trigger to trigger the Lambda and set it to trigger at a cadence of your choice (for example once per day, or once every 30 days).

AWS can also be configured to trigger Lambda from a number of other sources including email and SMS triggers. See Amazon's documentation on invoking Lambda for more options:

Invoking Lambda functions - AWS LambdaAWS Lambda

Configure For Sending Emails

In this example, we are sending an email with the report results. In order to enable the email, you will need to allow Lambda to access SES SendEmail/SendRawEmail service.

You will also need to create an IAM identity to enable email sending.

For more information on setting up email sending with AWS, see Amazon's documentation:

Send email using Lambda and Amazon SESAmazon Web Services, Inc.

In this example, we are sending an email with the report results. In order to enable the email, you will need to allow Lambda to access SES SendEmail/SendRawEmail service.

Next Steps

In this example, we run a report that combines the results of 2 Commander reports (security-report and security-audit-report) and send it via email as a JSON attachment, allowing us to obtain enterprise-wide security-monitoring data -- e.g., last-login date and overall security scores for each user in the enterprise -- in a regularly-scheduled and automated way. However, with this setup, any other set of Commander functions can be run with Lambda.

Experiment with other Commander functionality, Lambda invocation methods, and other AWS services (such as SNS for utilizing various methods for push notifications -- including SMS messages) to bring automated value to your Keeper processes.

For some examples of using the Commander SDK code, see the example scripts in the Commander GitHub repo:

Commander/keepercommander/custom at master · Keeper-Security/CommanderGitHub

To learn more about Commander's various methods, see the Command Reference section.

Command Reference