Practitioners Guide To Scaling IaC
Practitioners Guide To Scaling IaC
Infrastructure governance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 - 12
Technology future-proofing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 - 17
Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 - 19
About Spacelift . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Provisioning and managing modern cloud infrastructure is not easy. You need to provide fast,
secure, reliable infrastructure and keep pace with the intensifying technology demands of
modern applications. And you have to do all this while providing a self-serve, one-stop
infrastructure deployment platform for developers. The good news is that good infrastructure
makes huge financial sense: According to recent research by McKinsey1, the best-performing
infrastructure. Parallels with traditional code inspire many DevOps teams to attempt using a
generic CI/CD platform such as Jenkins, CircleCI, GitHub CI/CD, or GitHub Actions to manage
their IaC, but that approach creates various challenges as teams look to expand IaC adoption
In this guide, we examine four common challenges associated with using a generic CI/CD
Streamlining your
Infrastructure Developer velocity
Technology
We also provide an overview of purpose-built IaC management platforms that address these
Along the way, you’ll learn about the real-life experiences of DevOps professionals who have
3
Your options for IaC management
However, IaC does present challenges. The learning curve can be quite steep: Traditional
infrastructure or operations teams within organizations will need to familiarize themselves with
version control systems, Git, and code-editing tools like Visual Studio Code. As more members
of an organization adopt Terraform, infrastructure provisioning workflows have to change and
adapt. Workflows that function well for individual practitioners may not scale to larger
enterprises.
As organizations reach this point in their growth, they often turn to general-purpose CI/CD
tools like Jenkins, CircleCI, GitLab CI/CD, or GitHub Actions to manage IaC. They soon become
painfully aware that cloud infrastructure is complex and dynamic, and maintaining an IaC setup
becomes increasingly problematic in an expanding organization. As more teams are involved
with cloud provisioning, they have to agree on the best ways to use the chosen tool and learn
the nuances of each. Using IaC extensively across the organization and across multiple teams
makes it tricky to maintain traceability and versioning for all those configs. This complexity is
often what prompts organizations to adopt specialized IaC management platforms, sometimes
known as TACOS (Terraform Automation and Collaboration Software), to help them manage
IaC at scale, improving the developer experience without increasing risk.
4
“I tried to see if there was a way to do the centralization of
scale and you would run into issues in the long term.”
Luis Barrueco
IaC management platforms offer a centralized, collaborative way to manage Terraform code
and infrastructure. These platforms integrate with version control systems (VCS) and provide
specialized IaC functionality like remote state management, infrastructure policy enforcement,
and secure variable storage. They make it easier for organizations to maintain efficient control
over their cloud infrastructure by enabling teams to take care of infrastructure provisioning,
compliance, and management across various cloud providers, data centers, and services.
With purpose-built IaC management platforms, infrastructure can be scaled to meet increased
required to deliver infrastructure, embedding security and compliance requirements into the
5
“We have this great pipeline where we commit code and it comes
Alex Jurkiewicz
The best-known commercial options for managing your IaC are Hashicorp’s Terraform Cloud
(and its self-hosted version, Terraform Enterprise) and Spacelift. Later in the guide, we provide
IaC certainly offers clear benefits when it comes to efficient scaling, but it does introduce
complexity. Organizations need to understand the challenges it creates for DevOps and
platform teams and how to select the right management solution to address these pain points.
With IaC, you treat infrastructure the same way you treat application code, checking it into
version control, writing tests for it, and ensuring it’s consistent across multiple environments.
You have a choice of IaC tools to use for your workflow. For smaller operations requiring very
basic infrastructure, it might make sense to apply IaC locally — managing and running
As your operations scale, and you need to ship new applications more often, it doesn’t make
sense to manually provision and configure infrastructure for each new deployment.
6
That’s why you might turn to a generic CI/CD automation tool like Jenkins, which can be
customized to some extent for Terraform workflows. In fact, for organizations that have
adopted Jenkins for a whole range of tasks and who are still in the early stages of Terraform
Kévin Lemele
However, tools like Jenkins were designed to manage continuous integration and continuous
delivery for application development and are too generalized to build a complex workflow for
infrastructure deployment and management. With CI systems generally, it's a challenge to map
multiple deployment targets to a single codebase, and the problem is exacerbated by:
intricate dependencies between resources. Platform engineers get swamped with deployment
requests and must sift through infrastructure configuration files for issues that could cause
7
Specialized IaC management platforms
Streamlining complex IaC workflows is where specialized IaC management platforms shine.
They work via API, CLI, and UI, so they can be integrated easily into existing CI/CD pipelines, IT
service management interfaces, and VCS processes. They offer various features for achieving
end-to-end infrastructure orchestration using Terraform and other configuration languages.
These include:
Drift detection
With infrastructure, your actual state can deviate from your configuration. Most of this drift is
caused by manual intervention, so IaC management platforms provide drift detection and
optional remediation to address it as soon as possible.
Policy-as-code support
Policy-as-code frameworks automate the enforcement of compliance and governance policies
before any changes are made via Terraform. Leveraging code-based automation to manage
policies allows teams to move more quickly and reduces the potential for missteps due to
human error.
Impact analysis
By automating IaC workflows, specialized management platforms bring clarity to the potential
effects of proposed changes, ensuring continuous delivery for your teams.
Visualization tools
Specialized IaC tools help you gain a detailed understanding of your managed resources — not
just in their current context, but also from a historical perspective.
8
“To streamline our IaC process we built a strategy around policy
Timur Bublik
Leveraging features such as Spacelift’s stack dependencies, for example, you can build
Managing intricate dependencies is easy when you can define dependencies between stacks,
ensuring that the correct order is maintained when deploying or updating your resources. This
simplifies the process, removes bottlenecks, and facilitates the deployment of more modular
Infrastructure governance
cybercriminals, but also because the way it’s designed has implications for stability and
difficult to ensure compliance and security. If you are operating in a sector like finance, energy,
or healthcare, you have the added burden of onerous industry regulations to comply with.
One route to a solution is to increase automation, which has the added benefit of also
preventing audit failures. The ultimate aim is to implement continuous compliance automation,
9
For optimum infrastructure security, the approach you adopt needs to utilize the following
practices:
You can adapt general-purpose CI/CD tooling to ensure infrastructure governance from a
technical standpoint, but this results in a siloed approach and makes transparency virtually
impossible. With no visibility into the entirety of your infrastructure, it is difficult to maintain the
success.
Teams that use general-purpose CI/CD tools are fully responsible for their pipelines. Even with
shared libraries and other efforts to maintain communication, the team that establishes a new
pipeline must be aware of its configuration and confident of its security. Most generic IaC tools
lack access control models and depend on comments on pull requests to drive infrastructure
deployments, which introduces unacceptable risk the more complex your infrastructure
becomes.
Ultimately, managing security and compliance with general-purpose CI/CD tools is unwieldy
because they are not designed for infrastructure orchestration, but for the continuous
integration and delivery of application code to production systems. They approach automation
from a process perspective — they are not concerned about what code they ship.
10
Yes, it is possible to use general-purpose CI/CD tooling to orchestrate and govern your
infrastructure, but it will be a largely manual process and will require specialized expertise
Performing security and compliance checks using ticket-based review processes creates
bottlenecks that do nothing to advance any ambitions of accelerating developer velocity. That’s
why specialized IaC platforms use policy-as-code frameworks to automate the enforcement of
compliance and governance policies before any changes are made via Terraform.
Purpose-built IaC platforms like Spacelift use the open-source policy framework OPA across
the cloud-native stack, allowing you to define policies for Terraform and other supported tools.
Enforcing policies across all teams and infrastructure in this way means you can:
Some IaC management platforms allow you to integrate third-party security tools into your
workflows. With Spacelift, you either install and configure them directly into your runner hooks
or bring your own Docker image with the tools installed. A feature called Custom Inputs allows
you to define custom policies for the third-party input from these tools.
11
Other security and compliance measures available from these platforms include:
Role-based access control (RBAC), so you can restrict access to specific functionality based
on a user's role
Private workers, which ensure end-to-end encryption using your private key
SAML 2.0, following the single sign-on approach with centralized access to resources
Organizations must prioritize responsiveness to meet customer needs and adapt to market and
technology changes. This means accelerating the delivery of products and services that meet
these changing demands and requirements. Increasing developer velocity and productivity is
one way to ensure organizations can move faster. Seamless integration with existing CI/CD
pipelines, real-time monitoring and feedback, closing gaps between developers and DevOps
teams, self-service, and encouraging collaboration are some of the ways well-orchestrated
If you are applying your infrastructure through a generic CI/CD platform using a tool like
Terraform, you can see the many benefits it offers over traditional ClickOps approaches to
12
“Once we had the Spacelift platform installed and configured, it
faded into the background of our workflow, letting us operate more
efficiently with no hassle.
Alex Siegman
You can use generic CI/CD tools to help manage collaboration in a Git flow using GitOps — at
its most basic, through a Git repository where you share code or configuration information.
Depending on the type of data in question, you can share your data with other teams via read-
only access, but that approach means the other groups have to poll the repository for changes
because there is no notification system, and data flows one way only.
Alternatively, you could share a read/write repository and collaborate via pull requests. This is a
more authentic version of collaboration, but it requires close interaction among teams to ensure
With Jenkins, for example, you can coordinate pipelines on shared repositories by triggering a
job based on a change in a repo. You can also trigger the start of one job based on the
successful completion of another. This is not true collaboration, however, because when a team
completes a task that affects another, the completion of one job indicates only that it is finished;
no other information is conveyed. The triggering job must also be aware of the client job(s) that
need triggering.
13
If you are not using a specialized tool to manage your IaC operations, you are likely to
encounter the following obstacles to developer velocity and productivity:
1 Third-party plugins are required to extend Terraform's capabilities. These plugins can create
compatibility issues and require additional effort to manage.
2 Real-time monitoring and reporting are also difficult to achieve with general-purpose CI/CD tools.
They provide virtually no insights into the resource lifecycle from either a real-time or historical
perspective, but with IaC, it is vital to have a detailed understanding of managed resources, not
only each resource’s current status but also its historical context.
3 Although Terraform can be used to version control your infrastructure, it needs to be automated to
streamline conflict resolution.
4 Self-service is too risky to permit on any kind of scale if you are managing your infrastructure
through a generic CI/CD platform. IaC makes deploying infrastructure effortless, so it needs strict
guardrails to avoid costly mistakes.
The need to ensure security, accuracy, and consistency means that many organizations rely on a
small DevOps team as gatekeepers for their IaC operations. This ensures optimal protection
but it impedes collaboration and makes IaC administration very laborious. Terraform launched a
decade ago, so early adopters may have accumulated repositories spanning years' worth of
versions and approaches, adding dense layers of complexity and making it very stressful to
perform Terraform operations.
If DevOps teams have to wait for changes to take effect safely before they can move on to
anything else, productivity obviously suffers. Teams need automation and repeatable
processes. They need to be able to safely manage their own deployments and repositories
without DevOps engineers having to intervene every time a change is required. This is where
the specialized IaC management platform comes in.
14
Specialized IaC management platforms
A major driver and huge win associated with specialized IaC solutions is the way they
democratize IaC to the broader engineering organization. With the right guardrails and security
measures in place, organizations can confidently delegate much of IaC management to the
teams that own it. Teams can make the changes they need almost immediately, freeing DevOps
engineers to work on areas that benefit from their expertise — not just their value as IaC
gatekeepers.
Self-service
Collaboration
15
Promoting collaboration is a great way to encourage efficiencies and derive maximum
value from your teams, but there needs to be control. That’s where notification policies
come in. These are essential for auditing and managing your infrastructure deployments.
Notification policies allow you to create custom notifications that can be routed to the
notification inbox, Slack, or webhooks to deliver all the required details to make informed
deployments.
Control
Role-based access controls and policy enforcement are key to the secure yet streamlined
collaboration that specialized IaC platforms facilitate. Access control mechanisms enable
those with appropriate permissions to view or edit Terraform state, so you can encourage
team members to work independently toward the same goals — but with control.
Technology future-proofing
Technology moves at such a blistering pace that it is important to build your infrastructure
management strategy in a way that accommodates future change to the maximum extent
possible. It’s impossible to predict how your organization or the industry you operate in will
change, but you can anticipate shifts by being prudent about the way you approach your
infrastructure management and maximizing future flexibility by using agnostic tools as much as
possible.
16
It is relatively straightforward to deploy resources using Terraform through a generic CI/CD
platform from the same machine you’re developing the code on. All you need is access to the
target provider and an installed Terraform binary. However, once you have multiple DevOps
engineers working on a project, it becomes very complex, especially if you are not using remote
state. Even if you use something like pre-commit-terraform2 to run a set of selected tools to lint
the code you’re creating before you commit it, each action is manual and must be performed
directly by the developer. This makes scaling your infrastructure management a very clunky
process.
The general-purpose CI/CD tools you may be using to manage your infrastructure simply do
not enable the kind of nimbleness you need to future-proof your tech stack. The level of effort
required for every change makes it virtually impossible to update your technology to keep pace
with a shifting competitive landscape. Worse, they work only with the tools you use now. They
The inescapable reality is that IaC tools and frameworks come and go. Terraform, Ansible, and
CloudFormation may seem unassailable now, but Chef, Puppet, and SaltStack were the tools of
choice a decade ago — and where are they now? Pulumi, Crossplane, and OpenTofu are
contenders to become the new favored technologies, and yet more frameworks and tools will
emerge in the next five years. If your platform is designed to manage Terraform only, your
options are seriously limited if and when Terraform is superseded by an alternative IaC tool, or
if you decide to integrate a third-party tool or use multiple IaC languages. To future-proof your
technology, you need to ensure that the IaC management platform you choose today will
17
Conclusion
deployed, and managed. Tools such as Terraform drastically reduce the manual effort required
However, as the organization scales, managing your IaC through a generic CI/CD platform
becomes impractical — and ultimately almost impossible. The stress of running multiple users
and teams intensifies, and you may start to wonder what happened to the efficiency, control,
and ease of use IaC was designed to promote. As this expansion starts to exert unbearable
pressure on your DevOps and platform teams and stretches the capabilities of generic CI/CD
tools to breaking point, you need to find a better way. That’s when you know it’s time to adopt
“We had a lot of issues with setup, access control, and coordination
Chris Schafer
18
Choosing the right IaC management platform ensures you can scale securely and with the
maximum potential for efficiency, productivity, and insulation against future shocks. It allows
services to be designed, delivered, and operated safely and efficiently across on-premises,
cloud, and edge deployments. Your platform and DevOps teams can template service creation
and management without risk, providing strategic workflow capabilities to increase the speed
and quality of infrastructure services, enhance the traceability and visibility of service delivery,
Your tool of choice should deliver everything you need to easily build and maintain tailored,
multi-IaC workflows that can share outputs, detect and remediate drift, build self-service
infrastructure to drive developer velocity, and put you in a position where your long-term
infrastructure needs are assured. Implement the right IaC management system and free your
Now that you’ve decided to move to the next level with your infrastructure, which IaC
management platform should you choose? To help you decide, we compare two leading IaC
management platforms and the features they offer to help you streamline your workflows,
manage governance and complexity, boost developer velocity and productivity, and future-
19
Streamlining your IaC workflow
Why it matters
Full workflow control Full control over your Limited Enhances visibility,
workflow consistency, and security
20
Managing governance and complexity
Why it matters
Policies across various Control almost any Plan and approval Enhances control
detection points aspect of the policies only
platform through
policies.
21
Developer velocity and productivity
Why it matters
Self-service Blueprints
No-code Boosts efficiency
Kubernetes Operator provisioning
(modules)
Kubernetes Operator
Shareable contexts Yes (env vars, files, Yes — but env vars Enables secure
runner hooks) only collaboration
Integrations Yes (you can Yes — but run tasks Makes it easier for
integrate with any only developers to work with
tool across any task their favorite tools
in the IaC lifecycle)
Resource management Full inventory Resource view only Full visibility at a glance
management available per boosts productivity.
solution including workspace
visualization,
lifecycle tracking,
search, and filtering
22
Future-proofing your IaC
Why it matters
23
About Spacelift
lifecycle of your infrastructure. It integrates with your choice of VCS to access and manage your
infrastructure code across all IaC tools (e.g. Terraform, OpenTofu, CloudFormation, Pulumi).
By providing developer self-service, golden paths with guardrails, and an OPA policy engine,
Spacelift empowers businesses to accelerate developer velocity while maintaining control and
governance over their infrastructure. Spacelift offers unrivaled support, no-nonsense pricing,
Learn more about the Spacelift platform and how it can help you overcome your IaC challenges
at Spacelift.io. Sign up for a demo, or test the platform yourself with a free trial.
Developers
Infrastructure as Code
24
Author
Flavius Dinu
Flavius is a passionate Developer Advocate with an Infrastructure as Code
mindset and expertise in DevOps & Cloud Engineering. He holds ITIL Foundation
Certificate in IT Service Management and Hashicorp Terraform Associate
Certification. He currently works at Spacelift, and in his free time, he blogs
at techblog.flaviusdinu.com, where he provides tutorials, tips, and tricks for all
levels of experience based on his exposure.
References
1 Developer Velocity: How software excellence fuels business performance
McKinsey & Company
2 pre-commit-terraform
GitHub - Anton Babenko
25