Serverless File Transfer Workload – Part 1 – SFTP

Introduction

Suppose a file transfer workload exists between a business and their customers. A comma-separated values (CSV) file is transferred to the business and the records are loaded into a database. The business has regulatory requirements mandating that all external assets are virus scanned before being processed. Additionally, an intrusion prevention system (IPS) must operate on all public endpoints.

In the following 3 articles I will demonstrate how we can build a serverless system that meets these requirements.

Design

We will use the Secure File Transfer Protocol (SFTP) to enable the transfer of files between the customer and the business. Using the AWS Transfer Family service we can create an SFTP endpoint with an Amazon S3 bucket to store the files. An AWS Network Firewall will sit in front of our SFTP endpoint.

AWS Network Firewall is a managed service that makes it easy to deploy essential network protections for all of your Amazon Virtual Private Clouds (VPCs).

AWS Network Firewall’s intrusion prevention system (IPS) provides active traffic flow inspection so you can identify and block vulnerability exploits using signature-based detection.

For our Network Firewall deployment, we will follow the multi-zone internet gateway architecture as described at Multi zone architecture with an internet gateway – Network Firewall (amazon.com)

A simplified view of our infrastructure is shown below.

Design diagram for SFTP

Terraform

The Terraform code at aw5academy/terraform/sftp can be used to apply the infrastructure components.

Terraform apply console output

Make a note of both the bucket-name and sftp-endpoint outputs… we will use both of these values later.

With Terraform applied we can inspect the created components in the AWS console. Let’s first check our SFTP endpoint which can be found in the AWS Transfer Family service.

SFTP endpoint

We can also see the AWS Network Firewall which is in the VPC service.

AWS Network Firewall

Testing

Let’s test out our solution. First, in the root of the Terraform directory, an example.pem file exists which is the private key we will use to authenticate with the SFTP endpoint. Copy this to your Windows host machine so we can use it with WinSCP.

In WinSCP, create a new site and provide the sftp endpoint. For username we will use “example”.

WinSCP new site

Select “Advanced” and provide the path to the example.pem you copied over. It will require you to convert it to a ppk file.

WinSCP SSH

Now login and copy a file across.

WinSCP file copy

Lastly, verify the file exists in S3 from the AWS console.

S3 Console

Success!

Now let’s continue with part 2 where we will implement the anti-virus scanning.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s