aws-autoscaling-gitlab-runner
CloudFormation template to deploy a GitLab Runner with auto-scaling on AWS.
[!CAUTION] The Docker machine based auto-scaling executor for GitLab Runner is deprecated. This project is now unmaintained and archived.
We encourage you to migrate to the new fleeting plugins for GitLab Runner.
This repository consists of an AWS CloudFormation template that may be used to deploy a GitLab runner with Docker executor and auto-scaling based on number of builds on AWS.
The runners have a shared cache to speed up builds. Objects in the bucket are automatically expired after a configurable number of days — 0 means that cache objects will never expire.
Resources created
-
1 S3 bucket to store runners' cache.
-
1 EC2 instance that is the runners' manager: it invokes AWS APIs to spawn and terminate other EC2 instances (via
docker-machine
) and runs Docker containers on them to process GitLab CI builds.
VPC consideration
If you want to create the stack within a specific VPC, you'll first need to create it manualy.
Please note that docker-machine uses availability zone A per default therefore you'll need to have at minimum the 'A' zone available in your vpc.
- Note: This is something you can contribute to and propose a change request to add an availability zone selector...
Obtaining a GitLab Runner token
When you launch the stack you are required to pass a GitLab Runner token. This is not to be confused with a GitLab Runner registration token!
You can obtain a registration token by navigating to the "Settings › CI / CD" page of any project for which you have administrative rights. It'll be available under "Runners settings".
You can then obtain a GitLab Runner token by using the (undocumented) endpoint
POST /runners
:
# Assuming the GitLab instance is available at https://gitlab.example.org # and the GitLab Runner registration token is "abcdef1234567890": curl -XPOST -H 'Content-Type: application/json' -H 'Accept: application/json' \ -d '{"token":"abcdef1234567890","run_untagged":true,"locked":false}' \ https://gitlab.example.org/api/v4/runners
If everything goes fine, the response will be a JSON that has a token
key:
this is the GitLab Runner token you were looking for.
Security considerations
AWS credentials
Credentials must be rotated, and humans must remember to rotate credentials. But credentials are not always strictly necessary.
The runners' manager instance has an AWS Instance Profile attached that makes it possible to invoke EC2 and S3 APIs using dynamically obtained credentials, that have a short lifetime and therefore don't need to be rotated. Thus, no IAM access keys are involved in this stack — except the ones you may use to create or update the stack using the AWS APIs or the CLI, of course.
GitLab CI token
The only credential that is actually stored somewhere is the GitLab CI token. If stolen, it would allow a malicious user to "intercept" your builds and run them on their infrastructure, exposing other secrets as a consequence. You should treat this as a very sensitive information.
This stack doesn't provide any special security measure: the value is passed as
plain text to CloudFormation at stack creation as a "sensitive parameter"
(NoEcho: true
), and is stored in plain text on the runners' manager in a file
that is readable only by gitlab-runner
user. The provisioning of said file
happens via cfn-init
. The value is then used by GitLab Runner itself,
presumably in HTTP-over-TLS communications with the GitLab instance.
SSH
At stack creation you are required to specify an AWS Key Pair to provide access to the runners' manager instance. When the stack is created, you can access your instance with the following command:
ssh -i /PATH/TO/IDENTITY_FILE ec2-user@INSTANCE-PUBLIC-IP
The runners' manager Security Group allows connections on port 22 by any IPv4
address (CIDR: 0.0.0.0/0
), and all other ports are unaccessible. This is far
from being an optimal solution but, since SSH authentication requires an SSH key
pair, it should be pretty safe anyways. Counter-measures like Fail2Ban are
not deployed out of the box, either.