AWS Secrets Caching

On the 18th of October, AWS released a new Lambda extension to simplify and cache secrets and parameters: https://aws.amazon.com/about-aws/whats-new/2022/10/aws-parameters-secrets-lambda-extension/

This extension helps reduce the number of API calls a function needs to make to Secrets Manager and allows values to be cached locally for concurrent executions. Not only does this get around API rate limiting issues, but it can also drive down costs per lambda execution.

It works by hosting a local HTTP server within a Lambda Layer where your functions can make GET requests to retrieve secrets rather than relying on the AWS SDK.

We’re going to run through how to get up and running with this new extension using the Serverless Framework and NodeJS.

First, we will start by creating a new serverless project using the basic starter template:

When we open this in VScode we will have a directory structure like this:

We will start by adding the AWS Secrets extension as a new layer in ‘serverless.yml’

I am deploying to eu-west-1 so I have used the ARN that corresponds to this region, you can find the relevant ARN for your deployment here https://docs.aws.amazon.com/secretsmanager/latest/userguide/retrieving-secrets_lambda.html#retrieving-secrets_lambda_ARNs

We also need to configure the Lambda IAM role to allow access to secrets manager, I am going to set this quite loose, but you should be more selective in a production environment and scope the role accordingly.

Modify the ‘providers’ section like so:

We now have a Lambda function defined with Serverless, have attached the Secrets Manager extension and configured the required IAM permissions to fulfill its duties.

Next, we will create a Secret which we will use to test the extension using the AWS CLI:

And now we will configure our Lambda function to grab the secret using the extensions web service and output it to the console.

Update handler.js as follows:

The header: is used to pass the session token from the Lambda functions execution role into the extension so it can authenticate with Secrets Manager.

Now deploy the function with:

Finally, we can invoke the function and see the output of the cached secret:

And there you have it.