Create AMIs with AWS CodePipeline
by Henry Bravo on 01 JUN 2023 in Amazon EC2, AWS CloudFormation, AWS CodeCommit,
AWS CodePipeline, Technical How-to Permalink Comments Share
One way to alleviate the pain of drift in your Amazon Web Services (AWS) workloads is to use
a GitOps workflow to automate Amazon Machine Image (AMI) builds with AWS CodePipeline.
In this post, we show you how to deploy a continuous integration and continuous delivery
(CI/CD) pipeline, managed by CodePipeline, to build AMIs.
Manually creating AMIs for your Amazon Elastic Compute Cloud (Amazon EC2) instances and
containers can be an error-prone process. As your AWS workloads grow, you can lose track of
infrastructure changes and discrepancies, or drift. The solution I provide uses a Git repository
managed by AWS CodeCommit as the single source of truth. You invoke the pipeline when you
commit AWS CloudFormation templates containing the recipe that the pipeline uses to build an
AMI. The templates I provide build an AMI that’s based on Amazon Linux2 ARM64. After the
AMI is built, Amazon Simple Notification Service (Amazon SNS) sends you an email
notification. After you approve or reject the AMI build, the pipeline automatically cleans up the
resources it created.
About this blog post
Time to read ~10 min.
Time to complete ~45 min.
Cost to complete ~$1
Learning level Advanced (300)
AWS CodePipeline
AWS CodeCommit
AWS CloudFormation
AWS services Amazon Elastic Compute Cloud (Amazon EC2)
Amazon EC2 Image Builder
Amazon Simple Storage Service (Amazon S3)
Amazon Simple Notification Service (Amazon SNS)
Overview
Figure 1 shows a high-level overview of the resources that you deploy in the walkthrough.
Figure 1. Architecture diagram for creating AMIs with AWS CodePipeline.
As shown in Figure 1, the solution deploys the following:
A CD pipeline with the following:
o CodePipeline to read source code from AWS CodeCommit.
o CodeCommit with a GitHub repository. You invoke the pipeline when you
commit templates to the repository.
o CloudFormation to create an Amazon EC2 Image Builder stack based on the
template.
o Amazon SNS to send email notifications upon completion of AMI builds. The
notifications prompt you to review and approve or reject builds in the
CodePipeline console.
An Amazon Simple Storage Service (Amazon S3) bucket to store templates and artifacts.
EC2 Image Builder to launch a temporary Amazon EC2 T4g instance that builds and tests
an AMI.
AMI recipe template files
Our solution uses the following templates to define the AMI build recipe. The templates contain
the recipe used by Amazon EC2 Image Builder to build, validate, and test an AMI.
linux-ami-imagebuilder.yaml
The template linux-ami-imagebuilder.yaml contains the following EC2 Image Builder
parameters:
LinuxImage tells the EC2 Image Builder service that we are building an AMI and
specifies the image recipe and infrastructure configuration to use. The image recipe
defines how the AMI is customized, and the infrastructure configuration defines the build
and test configurations.
LinuxImageRecipe defines the components to configure and deploy to the AMI. You
can customize the solution by specifying different components, including your own
custom components that you write in Bash or PowerShell. I configure the template to
deploy the following components:
o amazon-cloudwatch-agent-linux: Amazon CloudWatch agent.
o aws-cli-version-2-linux: The AWS Command Line Interface (AWS CLI)
version 2.
o stig-build-linux-low: The Security Technical Implementation Guide (STIG)
for Category 1 (Low) compliance level.
o SetLocaleComponent: The setlocale function in Linux sets the current locale,
which determines the language and cultural conventions used by various system
functions and libraries. It fixes the following common error when logging into
Linux:
-bash: warning: setlocale: LC_CTYPE: cannot change locale (UTF-8): No such
file or directory.
In this warning, the LC_CTYPE category of the locale specifies character classification, case
conversion, and character encoding/decoding. When the LC_CTYPE category is set, it affects how
functions like isalpha() and toupper() behave. It’s important to note that setlocale affects
not only your own program but also any other programs running on the system that rely on the
locale.
o YumUpdateComponent—Invokes the yum update command. This command in
Linux updates the installed packages to their latest available versions. For more
information, refer to Linux package management with YUM and RPM.
stack-configuration.json
The template stack-configuration.json includes the parameters needed to run the pipeline.
Edit stack-configuration.json to specify the parameters, such as the base AMI image used
by the pipeline to build the AMI. You can customize your solution by entering the Value key
from a different AMI for the BaseImageId parameter. To find an AMI, refer to Find a Linux
AMI.
Prerequisites
Before getting started, ensure that you have the following:
An AWS account. If you don’t have one, sign up at https://aws.amazon.com.
A virtual private cloud (VPC) with a subnet in your AWS account.
An EC2 key pair. A key pair enables you to log in using SSH (Secure Shell) to the AMI
produced by the pipeline.
Version 2 of the AWS CLI. You need this if you want to deploy the CloudFormation
stack using AWS CLI instead of the AWS Management Console.
A Git client installation. You need this to use Git to invoke the pipeline by committing
templates. In the walkthrough, you can also use the AWS Management Console to
commit them.
Walkthrough
In the walkthrough, you perform the following steps:
1. Download the templates.
2. Deploy the CD pipeline CloudFormation stack.
3. Commit files to the CodeCommit repository.
4. Verify the AMI build.
5. Approve or reject the AMI build.
Step 1: Download the templates
Download the following templates:
linux-ami-codepipeline.yaml
linux-ami-imagebuilder.yaml
stack-configuration.json
Step 2: Deploy the CD pipeline CloudFormation stack
Deploy the pipeline infrastructure using the CloudFormation stack that I provide. You can
deploy using the AWS Management Console or AWS CLI.
Deploy using the AWS Management Console
1. Choose Launch Stack.
2. On the Specify stack details page of the CloudFormation console, enter a name for the
stack.
3. For Email, enter an email address to receive AMI build notifications.
4. (Optional) If desired, you can edit default entries except for TemplateFileName and
StackConfig.
5. Choose Next.
6. On the Configure stack options page, choose Next.
7. On the Review page, choose I acknowledge that AWS CloudFormation might create
IAM resources.
8. Choose Submit. The stack takes about two minutes to deploy.
Deploy using AWS CLI
In the directory where you save linux-ami-codepipeline.yaml, run the following command.
Replace <my@email-address.aws> with the address that you want to use to receive AMI build
notifications.
aws cloudformation create-stack[TB1] [MOU2] --region eu-west-1 --stack-name
ami-builder-blog-linux-pipeline --template-body[TB3] [MOU4]
file://Linux/linux-ami-codepipeline.yaml --parameters ParameterKey=
Email,ParameterValue=<my@email-address.aws> --capabilities
CAPABILITY_NAMED_IAM
Confirm CodePipeline SNS topic subscription
You will receive an email from AWS Notifications <no-reply@sns.amazonaws.com> asking
you to confirm subscription to the CodePipeline SNS topic. Choose Confirm subscription in the
email message.
Step 3: Commit files to the CodeCommit repository
Commit linux-ami-imagebuilder.yaml and stack-configuration.json to the
CodeCommit repository. This invokes the pipeline to build an AMI.
1. Sign in to the AWS Management Console and open the CodePipeline console.
2. In the navigation pane, choose Pipeline, Pipelines.
3. Choose ami-builder-blog-linux-pipeline. As shown in Figure 2, an error displays in the
Source section of the ami-builder-blog-linux-pipeline page. This is normal and
expected because you have not yet committed files to invoke the pipeline.
Figure 2. Ami-builder-blog-linux-pipeline page in the CodePipeline console.
4. In the Source section, choose AWS CodeCommit.
5. Set up your GitHub repository on the ami-builder-blog-linux-pipeline page. For Clone
URL, select Connection steps and follow the instructions provided. Then add linux-
ami-imagebuilder.yaml and stack-configuration.json to your repository and
commit them in a pull request to the CodeCommit repository.
If you’re not using Git, choose Add file, Upload file to manually upload linux-ami-
imagebuilder.yaml and stack-configuration.json one at a time. For more information,
refer to Create or upload a file (console).
Step 4: Verify the AMI build.
If you confirmed your subscription to the pipeline SNS topic, you will receive a notification
email when the build completes. To verify the AMI build, launch an instance of it and then log in
to the instance.
Note: If the build pipeline fails, terminate the Build EC2 instance and delete the temporary
CloudFormation stack test-amazon-linux2-arm64-ami that the pipeline deploys. Then
recommit the templates to restart the build process. To troubleshoot issues with the AMI build,
refer to the Amazon CloudWatch logs. The solution stores AMI build logs in
CloudWatch/Loggroups/aws/imagebuilder/test-amazon-linux2-arm64-ami-Image-
xxxxxxxxxxx/0.0.1/1/.
Step 5: Approve or reject the AMI build.
If you confirmed your subscription to the pipeline SNS topic, you will receive a notification that
includes a link to approve or reject the build. For more information, refer to Approve or reject an
approval action in CodePipeline.
Cleanup
After you approve or reject, the pipeline cleans up the resources it created, including the
CloudFormation template and EC2 Image Builder resources. When you’re finished testing the
pipeline and want to remove the solution from your account, delete the following:
The pipeline artifacts S3 bucket. For more information, refer to Deleting a bucket.
The CloudFormation stack ami-builder-blog-linux. For more information, refer to
Deleting a stack on the AWS CloudFormation console.
Conclusion
In this post, we deployed a CD pipeline managed by CodePipeline to build AMIs. Customize the
solution used in this post by adding unit testing to the CD pipeline and additional software and
operating system logic. For more insight into building AMIs, refer to the following:
AWS for Microsoft Workloads Immersion Day
Create immutable servers using EC2 Image Builder and AWS CodePipeline
Automate OS Image Build Pipelines with EC2 Image Builder
Let us know how you use CodePipeline to build AMIs. Use the Comments section for questions
and comments.
