Streamlining CI/CD Pipeline Setup with Docker Compose for GitLab Runner

Introduction

Managing Continuous Integration and Continuous Deployment (CI/CD) pipelines in a home lab can be both rewarding and challenging. As someone who has been using GitLab Runner in a self-hosted environment, I've experienced firsthand the intricacies of setting up and maintaining runners. Until recently, I'd been manually configuring each new GitLab Runner using Docker. However, I found a more efficient method by leveraging Docker Compose. In this article, I will share the insights I gained from transitioning to Docker Compose and provide a solution to streamline the process.

The Traditional Approach: Manual Configuration

Initially, my approach to setting up GitLab Runners involved manual configuration using Docker. Each time I needed a new runner, I meticulously configured it with the necessary settings and parameters. While this method worked, it was repetitive and time-consuming. As my requirements grew, the inefficiencies of manual configuration became more evident.

Challenges with Manual Configuration

1. Repetition: Every new runner required the same set of commands and configuration, leading to redundancy.
2. Human Error: Manual setups are prone to errors, especially when multiple steps are involved.
3. Maintenance: Managing updates or changes across multiple runners became cumbersome.

The Realization: Automation with Docker Compose

One day, I realized that there had to be a more streamlined approach. Docker Compose presented itself as a powerful tool for automating multi-container Docker applications. By defining a single configuration file, setting up multiple GitLab Runners could be handled with ease.

Benefits of Docker Compose

1. Simplicity: A single YAML file captures all configuration settings.
2. Reusability: The same file can be used to set up multiple runners, eliminating redundancy.
3. Scalability: Adding new runners or making configuration changes is straightforward.

Creating a Docker Compose File

To implement this solution, I created a Docker Compose file that performs two main tasks:

1. Creates the GitLab Runner Service: Configures and starts the GitLab Runner container.
2. Registers the Runner: Launches an auxiliary container to handle the registration process, which only runs once to register the runner with the GitLab CI instance.

The Docker Compose file can be found in my GitHub repository here.

Below is the Docker Compose file looks like:

services:
  gitlab-runner:
    volumes:
      - gitlab-runner-volume:/etc/gitlab-runner
      - /var/run/docker.sock:/var/run/docker.sock
    container_name: ${CONTAINER_NAME}
    restart: unless-stopped
    image: gitlab/gitlab-runner:latest
  
  gitlab-runner-setup:
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /usr/bin/docker:/usr/bin/docker
    restart: "no"
    image: ubuntu:latest
    depends_on:
      - gitlab-runner
    entrypoint: ["bash","-c","/usr/bin/docker exec ${CONTAINER_NAME} gitlab-runner register --non-interactive --url 'https://gitlab.com/' --token \"$TOKEN\" --executor \"docker\" --docker-image alpine:latest --description \"docker-runner\""]

volumes:
  gitlab-runner-volume:

Environment Variables

To facilitate easier modifications, the Docker Compose file leverages environment variables for essential parameters like TOKEN and CONTAINER_NAME. These variables can be defined in a .env file:

# Name of our Gitlab Runner Container
CONTAINER_NAME=""
# Gitlab Runner Auhtentication Token
TOKEN=""

Running the Setup

To set up the GitLab Runner:

1. Clone the repository:

   git clone https://github.com/ajohnsc/gitlab-runner-docker-compose.git
   cd gitlab-runner-docker-compose
   

2. Copy and modify the .env file with the necessary environment variables:

   mv .env.example .env
   

3. Start the services using Docker Compose:

   docker compose up -d
   

Once the runner is registered, the gitlab-runner-setip service will exit, while the gitlab-runner service will keep running.

Conclusion

Transitioning from manual configuration to Docker Compose for setting up GitLab Runners has vastly improved efficiency and reduced the likelihood of errors in my home lab. By automating the process, I can now spend more time focusing on actual development and testing rather than managing infrastructure.

I encourage anyone running GitLab Runners in a self-hosted environment to consider adopting Docker Compose. It simplifies configuration management, scales effortlessly, and brings consistency to your CI/CD pipeline setup. For the complete Docker Compose setup file, please visit my GitHub repository.

Happy automating!