Serverless Caching With AWS AppConfig and Lambda Extensions

Introduction

In this article I will show how you can deploy a simple caching solution for AWS Lambda functions by combining the AWS AppConfig service with the Lambda Extensions feature.

To demonstrate this, lets create a problem that we must solve. Suppose you have been asked to implement a solution that will allow the engineers on your team to query any IPv4 address to check if it is in the AWS IP address ranges.

Design

Our solution to this problem will be very straightforward. We will have an Amazon S3 bucket serving a single HTML page which, allows the user to input an IP to check. This rudimentary web application will make a request to an Amazon API Gateway REST API. The API will use Lambda to check the input IP against the list of Amazon IP ranges. The IP ranges will be stored in AWS AppConfig.

Next, we will take advantage of the AppConfig Lambda extension so that our function does not need to call AppConfig on every invocation.

Lambda Extension

To use the AppConfig Lambda extension, we first attach a Lambda layer to our function code. The documentation here provides all of the per-region ARNs for the AppConfig Lambda extension.

Once attached, we modify our function code to make a request to a localhost HTTP endpoint that is created by the layer. This endpoint will regularly poll AppConfig for your configuration data and maintain a local cache of it which is available to your function.

Your function code can then query the endpoint for the configuration data. Some sample code showing this in use would be:

import json
import urllib.request

def lambda_handler(event, context):

  app_config_app_name = "foo"
  app_config_env_name = "live"
  app_config_profile_name = "data"

  url = f'http://localhost:2772/applications/{app_config_app_name }/environments/{app_config_env_name}/configurations/{app_config_profile_name}'
    config = json.loads(urllib.request.urlopen(url).read())

SAM

The AWS Serverless Application Model (SAM) code at gitlab.com/aw5academy/sam/awsip can be used to deploy this solution.

This code defines:

  • A lambda function;
  • The AppConfig Lambda extension layer;
  • An API Gateway API;

Deploy the code with:

git clone https://gitlab.com/aw5academy/sam/awsip.git
cd awsip
sam deploy --guided

When deployed, the API endpoint will be output. We will need this value in the next section.

Web Application

Let’s now create an S3 bucket to host our web application. From the S3 service, create a bucket and untick the Block all public access checkbox and acknowledge the warning.

In your checkout of the sam/awsip repository you will see a HTML document at files/index.html. Edit this file and replace the variable value with the value output in the previous step.

var apiBaseUrl = "REPLACE_ME_LATER_AFTER_SAM_DEPLOY"

Next, upload this file to your bucket. Expand the “Permissions” section and tick the “Grant public-read access” radio button and acknowledge the warning.

AppConfig

We now need to deploy the IP data to AppConfig. In your checkout of the sam/awsip repository there is a Python script at scripts/create-app-config.py. Run this script to load the AWS IP ranges into AppConfig.

python3 scripts/create-app-config.py

We can confirm the script has worked by seeing the “1” and “2” configuration profiles under the “awsip” application in the AppConfig console.

Testing

We can now test our solution. In the S3 console, open the index.html object we uploaded and open the link under “Object URL”. Lets try an IP that we know does not exist in the AWS ranges.

Great! That works! Now let’s also check for one that does exist in the AWS range. (52.94.76.1 can be used).

Success!

Confirming Caching

Our solution works but can we verify that we are seeing a performance improvement by using the Lambda extension?

Earlier when you ran sam deploy, two outputs were the function ARNs. The first, “CheckIpFunctionArn” is the function attached to our API which contains the Lambda extension feature. The second, “AppConfigCheckIpFunctionArn” is a separate function that does not have the Lambda extension and instead makes a request to AppConfig directly for the configuration.

In your checkout of the sam/awsip repository you will see a Bash script at scripts/lambda-metrics.sh. Run this script and provide the name of the first function. E.g.

bash scripts/lambda-metrics.sh awsip-CheckIpFunction-S7XVscISBk2q

This script will invoke our function and report how long the function ran for. It will invoke the function 20 times and report the non cold start average duration.

So we see an average duration of approx. 897ms. Lets now try with the other function.

bash scripts/lambda-metrics.sh awsip-AppConfigCheckIpFunction-H2quZz4TWC28

We see now that the average duration is 959ms. So our caching saves us approx. 60ms.

Wrap-Up

This very simple solution implements serverless caching by using AWS AppConfig as a data source.

You can clean-up the resources by deleting the CloudFormation stacks created by SAM and you may also use the Python script at scripts/cleanup-app-config.py to remove the AppConfig resources.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s