Access Private EC2 Instances With AWS Systems Manager Session Manager

In this article I will demonstrate how you can connect to EC2 instances located in private subnets by using AWS Systems Manager Session Manager.

Session Manager is a fully managed AWS Systems Manager capability that lets you manage your EC2 instances, on-premises instances, and virtual machines (VMs) through an interactive one-click browser-based shell or through the AWS CLI. Session Manager provides secure and auditable instance management without the need to open inbound ports, maintain bastion hosts, or manage SSH keys.

https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager.html

Terraform

The Terraform code at aw5academy/terraform/session-manager will provision the resources for us.

Deploy the stack by issuing the following commands:

git clone https://gitlab.com/aw5academy/terraform/session-manager.git
cd session-manager
terraform init
terraform apply

Post Apply Steps

Some things are not configured in Terraform and must be set manually. These are the Session Manager preferences. To set these:

• Login to the AWS console;
• Open the Systems Manager service;
• Click on ‘Session Manager’ under ‘Instances & Nodes’;
• Click on the ‘Preferences’ tab;
• Click ‘Edit’;
• Enable KMS Encryption and point to the alias/session-manager key;
• Enable session logging to S3 bucket ssm-session-logs... with encryption enabled;
• Enable session logging to CloudWatch log group /aws/ssm/session-logs with encryption enabled;
• Save the changes;

Session Manager Plugin

To be able to use Session Manager from the AWS CLI you also need to install the Session Manager Plugin.

Start Session

Let’s try it out. First, we will use the AWS CLI to launch a new EC2 instance in the private subnet that was created by the Terraform code. This instance will have no key pair and will use the VPC’s default security group which allows no inbound traffic from outside the VPC.

aws ec2 run-instances \
    --image-id $(aws ssm get-parameters --names /aws/service/ami-amazon-linux-latest/amzn2-ami-hvm-x86_64-gp2 --query 'Parameters[0].[Value]' --output text --region us-east-1) \
    --instance-type t3a.nano \
    --subnet-id $(terraform output private-subnet-id) \
    --iam-instance-profile Name=session-manager \
    --output json \
    --region us-east-1 \
    --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=session-manager-test}]' \
    --count 1 > /tmp/ssm-test-instance.json

Next, run this command to wait for the instance to become ready:

while true; do if [[ $(aws ssm describe-instance-information --filters "Key=InstanceIds,Values=`cat /tmp/ssm-test-instance.json |jq -r .Instances[0].InstanceId`" --region us-east-1 |jq -r .InstanceInformationList[0].PingStatus) == "Online" ]]; then echo "Instance ready." && break; else echo "Instance starting..." && sleep 5; fi; done

When the instance is ready we can connect to it with session manager:

aws ssm start-session --target $(cat /tmp/ssm-test-instance.json |jq -r .Instances[0].InstanceId) --region us-east-1

That’s it! You are now connected to a private EC2 instance in your VPC which, has no public IP, no key pair and no inbound access from outside the VPC defined in its security group.

Port Forwarding

As well as starting a shell session on an instance you can also use session manager to start a port forwarding session. Suppose you have an EC2 instance with a tomcat server running on port 8080, you could start a port forwarding session that maps local port 18080 to the instance’s port of 8080:

aws ssm start-session --target <INSTANCE_ID> \
                      --region us-east-1 \
                      --document-name AWS-StartPortForwardingSession \
                      --parameters "localPortNumber=18080,portNumber=8080"

You could then access the tomcat server via http://localhost:18080 on your workstation.

Restricting Access

You can create IAM policies to define who can access which instances. For example, the following policy will permit session manager access to instances that are not tagged with Team=admins:

{
  "Version":"2012-10-17",
  "Statement":[
    {
      "Effect":"Allow",
      "Action":[
        "ssm:DescribeInstanceInformation",
        "ssm:StartSession"
      ],
      "Resource":"arn:aws:ec2:*:*:instance/*",
      "Condition":{
        "StringNotEquals":{
          "ssm:resourceTag/Team": [
            "admins"
          ]
        }
      }
    },
    {
      "Effect":"Allow",
      "Action":[
        "ssm:StartSession"
      ],
      "Resource":[
        "arn:aws:ssm:*::document/AWS-StartSSHSession"
      ]
    }
  ]
}

Run As

By default, session manager sessions are launched via a system-generated ssm-user. We can change this by launching the session manager preferences, checking the `Enable Run As support for Linux instances option and providing the alternative user.

Now when we start a session we are logged in as this user:

Additionally, you may add a tag to IAM roles or users with the tag key being SSMSessionRunAs and the tag value being the user account to login with. This allows you to further control access to your EC2 instances. See here for more details on this.

Summary

I hope this article demonstrates both how useful session manager is and how easy it is to setup and configure. Beyond the advantages described above you also get a full log of all sessions delivered to a CloudWatch log group and an S3 bucket for auditing purposes. These are configured in the Terraform code I have provided.