Article Content
What Is Terraform?
Terraform, developed by HashiCorp, is an open-source infrastructure as code software tool. It provides a consistent approach to provisioning, changing, and managing your infrastructure. Infrastructure can mean anything from physical servers to containers or SaaS products. Terraform treats all these infrastructural components as resources, and it works by creating an execution plan describing what it will do to reach the desired state of these resources.
An important aspect of Terraform is its provider ecosystem. Terraform Providers are plugins that Terraform uses to interact with APIs for various services. This allows Terraform to manage a broad spectrum of resources including public cloud providers, cloud automation systems, container orchestration platforms, Continuous Integration/Delivery (CI/CD) tools, monitoring systems, networking tools, and more.
What Is GitOps?
GitOps is a methodology that applies the principles of Git, a widely-used version control system, to infrastructure and operations. The core idea behind GitOps is that Git serves as the single source of truth for declarative infrastructure and applications. With Git at the center of the delivery pipelines, every change is versioned, traceable, and auditable.
In GitOps, the desired state of the infrastructure is stored in a Git repository. Operations teams use pull requests to make changes, which are then automatically applied to the system. This approach ensures consistency and reliability in deployment processes and reduces the chances of human error. Tools like ArgoCD are commonly used in GitOps to sync the state in the Git repository with the state in the environment, typically Kubernetes.
GitOps promotes collaboration and visibility among teams. Developers and operators can see the entire history of changes, discuss modifications, and roll back to previous states if necessary. This way, GitOps bridges the gap between development and operations, fostering a more integrated and efficient workflow.
Why Use Terraform for GitOps?
Terraform’s declarative nature aligns well with the GitOps principle of storing the desired state of infrastructure as code. This combination enhances automation, consistency, and reproducibility in infrastructure management. Using Terraform in a GitOps workflow offers several advantages:
- Terraform defines infrastructure using a high-level configuration language: This code can be version-controlled in Git, making it easy to track changes, review updates, and maintain a historical record of the infrastructure state. In a GitOps model, when a pull request is merged, an automated process can trigger Terraform to apply the changes, ensuring that the actual state of the infrastructure matches the desired state.
- Terraform can manage multi-cloud environments: This means a GitOps setup with Terraform can handle diverse infrastructure providers with a single workflow, making it easier to maintain complex systems. This unified approach reduces complexity and the risk of errors, particularly in large-scale environments with multiple cloud services.
- Terraform’s plan and apply stages can be applied to GitOps pipelines: The plan stage provides a preview of changes, which can be reviewed as part of the pull request process. The apply stage is then executed automatically to make the actual changes, ensuring a controlled and predictable deployment process. This integration streamlines operations and enhances the security and stability of the infrastructure.
How to GitOps Your Terraform in 4 Steps
Here are the high-level steps you will need to take to use Terraform in a GitOps model, which some have called “GitOpsing Terraform”.
1. Set Up a Git Repository to Store Your Terraform Code
To begin, establish a Git repository to store your Terraform configurations. This repository acts as the central location for your infrastructure code, enabling version control and collaboration. Choose a Git hosting service like GitHub, GitLab, or Bitbucket, and create a new repository specifically for your Terraform code.
When setting up your repository, consider organizing your Terraform configurations into directories that reflect your infrastructure’s logical structure. For instance, you might have separate directories for network, compute, and storage resources. Include a README file to explain the repository structure and any guidelines for contributing to the codebase. It’s also advisable to implement branch protection rules to prevent direct pushes to the main branch and enforce code review practices through pull requests.
2. Configure Your Infrastructure Using Terraform Code
The next step is to configure your infrastructure using Terraform code. Terraform uses a declarative language, which means you specify what your infrastructure should look like, and Terraform will make it so.
First, you need to write a Terraform configuration file. This file describes the resources that make up your infrastructure. For example, it might specify a virtual machine, a database, and a network. Terraform uses providers to interact with cloud providers—you will need to specify a provider for your infrastructure. For example, if you’re using AWS, you will use the AWS provider.
Once you’ve written your configuration file and specified your provider, you can use the terraform init command to initialize your Terraform working directory. This command downloads the necessary provider plugins and sets up the backend for storing your Terraform state.
Note: You should only run terraform init once.
3. Create a Pipeline That Uses Terraform to Apply Changes to Your Infrastructure
After configuring your infrastructure with Terraform code, the next step is to create a pipeline that uses Terraform to apply changes to your infrastructure.
A pipeline is a series of steps that are executed in order to transform your code from source to a final deployable outcome. In our case, the final outcome is the updated infrastructure that reflects the changes in our Terraform code.
You can use any continuous integration/continuous delivery (CI/CD) tool to create your pipeline. Some popular choices are Jenkins and Codefresh. The general process is to define a pipeline configuration file, which specifies the steps of your pipeline. At a minimum, your pipeline should include steps to checkout your code, initialize Terraform, plan your changes, and apply your changes.
4. Use Pull Requests to Manage Changes to Your Infrastructure
Once you create your pipeline, you can start using pull requests to manage changes to your infrastructure. Pull requests are a way to propose changes to a codebase. They allow you to review, discuss, and eventually merge changes into the main codebase.
When you want to make a change to your infrastructure, you can create a new branch in your Git repository. Then, you can make your changes in this branch. Once you’re done, you can create a pull request to propose your changes.
The pull request will trigger your pipeline, which will apply your changes in a safe and controlled way. If the pipeline succeeds, you can merge the pull request to apply your changes to the main codebase. The pipeline will then use Terraform to apply your changes to the live environment.
An Example: Creating a GitOps Pipeline with Codefresh and Terraform to Make Changes in AWS
First, install the Codefresh CLI using npm installer using the command npm install -g codefresh.
Now let’s see how to create a GitOps pipeline with Codefresh and Terraform. First, create a codefresh.yml file, which defines your pipeline steps.
You should place this file in the root directory of your Git repository containing the Terraform code. The pipeline file could look like this:
version: '1.0'
stages:
- "clone"
- "init"
- "plan"
- "apply"
steps:
main_clone:
title: "Cloning main repository"
type: "git-clone"
repo: "your-github-user/your-repo"
revision: "master"
stage: "clone"
terraform_init:
title: "Terraform Init"
stage: "init"
image: hashicorp/terraform:latest
commands:
- terraform init
terraform_plan:
title: "Terraform Plan"
stage: "plan"
image: hashicorp/terraform:latest
commands:
- terraform plan
terraform_apply:
title: "Terraform Apply"
stage: "apply"
image: hashicorp/terraform:latest
commands:
- terraform apply -auto-approve
when:
branch:
only:
- master # or any branch you preferThis pipeline uses the following steps:
- main_clone: This step clones your repository.
- terraform_init: This step initializes the Terraform environment.
- terraform_plan: Here, Terraform will create an execution plan.
- terraform_apply: This final step applies the changes. The -auto-approve flag automatically applies the plan. For production environments, it might be safer to require manual approval.
To allow this pipeline to make changes in your AWS environment, you will need to integrate with AWS. Ensure that Codefresh has access to your AWS credentials. You can set this up as environment variables in Codefresh or use an IAM role if Codefresh runs within AWS.
Now, you can configure the pipeline to trigger on push to specific branches, such as master, or on pull request creation. When developers raise pull requests with Terraform changes, and they are reviewed and merged into the main branch, the pipeline automatically applies these changes to AWS.