MultiStage Ops is a CI/CD pipeline designed to automate the deployment of applications using Docker containers across multiple environments: test and prod. This project leverages GitHub Actions for CI/CD, AWS EC2 instances as self-hosted runners, Docker for containerization, and an Application Load Balancer for routing traffic to different environments.
- Project Overview
- Features
- Architecture
- Getting Started
- Installation
- Configuration
- Usage
- Domains and Routing
MultiStage Ops automates the process of building, testing, and deploying Dockerized applications across multiple environments using a branch-based GitHub Actions workflow. It ensures changes are deployed based on the branch they are pushed to:
master: Deploys to the production environment.test: Deploys to the testing environment.
The project integrates with AWS EC2 instances, Docker Hub, and an Application Load Balancer to manage traffic between environments.
- Multi-Stage Deployment: Automatic deployment of applications to different environments based on GitHub branch.
- Self-Hosted Runners: Uses EC2 instances as self-hosted runners for efficient and customizable CI/CD workflows.
- Dockerized Application: Builds, pushes, and deploys Docker images for each environment.
- Automated SSL and Domain Configuration: Integrates AWS Route 53 and Load Balancer to route traffic to appropriate environments using subdomains.
- CI/CD: GitHub Actions.
- Containers: Docker and Docker Hub.
- Hosting: AWS EC2 instances for self-hosted runners.
- Traffic Management: AWS Application Load Balancer with Route 53 for domain management.
Before setting up the project, ensure you have the following:
- AWS Account with access to EC2, ALB, and Route 53.
- GitHub Repository.
- Docker Hub Account.
- EC2 Instances (Test and Prod) with SSH access and Docker installed.
- Domains registered with Route 53 or other DNS providers.
- Git
- Docker
- AWS CLI
- SSH client for EC2 access
git clone https://github.com/your-username/multi-stage-ops.git
cd multi-stage-ops- Provision two EC2 instances (for
testandprod) in AWS. - Ensure they have:
- Docker installed.
- Internet access.
- SSH access configured.
Follow these steps to install the GitHub Actions runner on each EC2 instance.
-
SSH into your EC2 instance:
ssh -i path-to-your-key.pem ec2-user@your-ec2-public-ip
-
Download and set up the GitHub Actions runner:
mkdir actions-runner && cd actions-runner curl -o actions-runner-linux-x64-2.309.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.309.0/actions-runner-linux-x64-2.309.0.tar.gz tar xzf ./actions-runner-linux-x64-2.309.0.tar.gz
-
Configure and start the runner:
./config.sh --url https://github.com/your-org/your-repo --token YOUR_RUNNER_TOKEN ./svc.sh install sudo ./svc.sh start
Go to Settings > Actions > Runners in your GitHub repository and add appropriate tags:
self-hosted-testfor the test environment.self-hosted-prodfor the prod environment.
Navigate to your repository's Settings > Secrets and add the following secrets:
DOCKERHUB_USERNAME: Your Docker Hub username.DOCKERHUB_TOKEN: Your Docker Hub access token.
In .github/workflows/deploy.yml, configure the workflow to trigger on test and master branches:
on:
push:
branches:
- master
- testUpdate the workflow to use the appropriate self-hosted runners for the test and prod environments.
- Create a Route 53 Hosted Zone for your domain.
- Set up an Application Load Balancer in AWS to route traffic to
mydomain.org(prod) andtest.mydomain.org(test). - Configure DNS records to point to the load balancer.
Once everything is set up, you can start deploying your application:
- Test Environment: Push changes to the
testbranch to deploy to the test environment (URL:test.mydomain.org). - Production Environment: Push changes to the
masterbranch to deploy to the production environment (URL:mydomain.org).
The GitHub Actions workflow will automatically:
- Build and push the Docker image to Docker Hub.
- Pull the image on the corresponding EC2 instance.
- Deploy the container using Docker and Nginx.
- Production:
mydomain.org - Testing:
test.mydomain.org
Both subdomains are routed through an AWS Application Load Balancer and served by Docker containers running on EC2 instances.
You can integrate AWS Certificate Manager (ACM) to generate and attach SSL certificates to your Load Balancer for secure HTTPS traffic.