No server is easier to manage than no server.Werner Vogels, CTO @ Amazon
Managing a fleet of EC2 instances for your Jenkins slaves is cumbersome and time consuming, even when baking the configuration into an Amazon Machine Image (AMI). By combining AWS serverless products we can run an instance of Jenkins with substantially less overhead.
We will run our Jenkins master node in an AWS Fargate cluster. The JENKINS_HOME will be stored on an Amazon Elastic File System. We won’t have Jenkins slaves but will instead run jobs on AWS CodeBuild using the Jenkins plugin.
The Terraform code at https://gitlab.com/aw5academy/terraform/sls-jenkins can be used to provision the components we need. We can utilise this code as follows:
git clone https://gitlab.com/aw5academy/terraform/sls-jenkins.git cd sls-jenkins terraform init terraform apply
Once applied, we get the following:
Wait a few moments for the ECS task to fully start then open the jenkins-url output in your browser. You should see the Unlock Jenkins page:
We can obtain the password from the task logs.
However, let’s take advantage of a new feature of Fargate called ECS Exec. With this feature we can start a shell session in any container without requiring SSH ports to be opened or authenticating with SSH keys. To use this feature, ensure you have the latest version of the AWS CLI as well as the latest version of the session manager plugin.
Find the task id of the sls-jenkins task in the ECS console and use it with command:
aws ecs execute-command \ --region us-east-1 \ --cluster sls-jenkins \ --task <task-id> \ --container sls-jenkins \ --command "/bin/bash" \ --interactive
You can then find the password in the /mnt/efs/secrets/initialAdminPassword file.
Use the value to login to Jenkins and complete the setup wizard.
We will run Jenkins jobs in AWS CodeBuild.
AWS CodeBuild is a fully managed continuous integration service that compiles source code, runs tests, and produces software packages that are ready to deploy. With CodeBuild, you don’t need to provision, manage, and scale your own build servers.AWS CodeBuild – Fully Managed Build Service (amazon.com)
First, install the CodeBuild plugin.
Next, create a new pipeline job. You can use the sample project at https://gitlab.com/aw5academy/codebuild/sample-project.git as the source of the project.
The Jenkinsfile in this sample project starts a build of the sls-jenkins-small CodeBuild project. When we run the build we get the following output:
The logs from CodeBuild are pulled into Jenkins and displayed in the console output.
To verify our Jenkins configuration will persist, lets stop the ECS task.
And if we open Jenkins in our browser we see an outage as expected.
ECS will now launch a new task and will remount the EFS file system that stores our JENKINS_HOME. And if successful we will see the sample-project job that we created earlier.
This solution may be a good fit for very simple Jenkins implementations. You will find that the EFS performance is not as good as EBS or ephemeral storage. There is also a queueing and provisioning time for CodeBuild which you would not experience with your own fleet of EC2 instances. These factors should be considered but if you spend a lot of time maintaining your CI/CD infrastructure, this solution could be useful to you.