Create Git Repositories and CI/CD Pipelines with the Terraform Provider for Codefresh

Create Git Repositories and CI/CD Pipelines with the Terraform Provider for Codefresh

8 min read

The Codefresh User Interface allows all application stakeholders to view and manage their applications in a user-friendly manner with comprehensive dashboards and detailed overview screens. It is not however the only way of managing Codefresh. Codefresh also comes with a powerful CLI as well as an extensive API that allows developers and operators to completely bypass the UI and create their own automated workflows with their favorite tools.

For Terraform users, a popular way to manage Codefresh is with Terraform. The Codefresh-Terraform provider (fun fact: the first version was actually created by a customer) allows you to handle Codefresh resources in a declarative way. Terraform is the de-facto industry standard for handling infrastructure as code. While Terraform is typically used for managing cloud resources such as VMs, clusters, load balancers etc, in reality it can handle any external resource as long as there is a provider for it.

This means that people who don’t want to use the Codefresh UI for managing stuff have an alternative way to integrate their Codefresh resources in their existing terraform setup.

Quick start: create a Codefresh project with Terraform

Let’s see a quick example. We want to group several pipelines inside a Codefresh project. Instead of creating the project by clicking a button in the Codefresh UI we will do it with a terraform file.

First let’s create a terraform file:

terraform {
  required_providers {
    codefresh = {
      source = "codefresh-io/codefresh"
      version = "0.3.1"

provider "codefresh" {
  api_url = var.api_url
  token = var.token

resource "codefresh_project" "test" {
  name = "myproject"

  tags = [

This terraform file says that we will use the Codefresh provider (version 0.3.1) that needs two variables to work (token and api_url). We define a single resource – a project named “myproject” with the tag “docker”.

To authenticate against Codefresh you need a token as an environment variable called CODEFRESH_API_KEY.
For the token variable you need to create a Codefresh token from your user settings:

How to create a Codefresh token
How to create a Codefresh token

If you are not using the SAAS version of Codefresh you also need to provide CODEFRESH_API_URL with your on-prem Codefresh URL. Both variables are also available as Terraform inputs.

Once everything is ready run

terraform init
terraform apply

Now if you go inside the Codefresh UI you will see the project!

Codefresh project created by Terraform
Codefresh project created by Terraform

Using the Codefresh Terraform provider is very easy and is fully compliant with all Terraform workflows. You can run “terraform destroy” and after a confirmation the project will be deleted inside Codefresh.

Real world scenario: Create your CI pipelines with Terraform

Creating a Codefresh project with Terraform is fairly simple, but not very exciting or realistic. A more natural scenario would be to create Codefresh pipelines in a dynamic manner.

We have already explained how to create Codefresh pipelines programmatically using the Codefresh CLI (part 1 and part 2).

We can do the same thing with Terraform as the Codefresh provider supports natively pipelines as a resource. To take this one step further we will solve the real scenario of project bootstrapping.

If you work in a company with lots of developers you will know what this problem is about:

Doing work via tickets and manual steps
Doing work via tickets and manual steps

Essentially whenever a developer creates a new application, somebody must create the respective git repository, pipeline and Codefresh entities that will allow them to build the project.

Depending on your automation status, this workflow can often be slow and cumbersome especially if the request must happen via a ticketing system.

Ideally we should automate the whole workflow so that no manual steps are involved. Apart from the Codefresh provider we can also take advantage of the GitHub provider. The GitHub provider allows Terraform to create GitHub repositories and files using a declarative format.

We will thus automate everything using the two providers as below:

  1. A “template” Git repository will be created once in GitHub
  2. A “best practices” pipeline will be stored with this Git repository along with skeleton source code
  3. The GitHub provider will be used to create a new GitHub repository by coping the template one
  4. A Codefresh project will be created
  5. A Codefresh pipeline will be created. The definition of the pipeline will be loaded from the newly created GitHub repo
  6. A pipeline trigger will be created that responds to GitHub commits and runs the pipeline.

Once the whole process is automated, we can run a single Terraform command for everything and respond to the developer as fast as possible.

Self-service developers
Self-service developers

First let’s create the template GitHub project that will be used as the archetype for any new GitHub repos. This depends on what your organization needs and ideally all developers should have access to this in order to maintain it as they see fit.

In my case, I created a starter Java/Spring project. The repository contains a sample Dockerfile and a codefresh.yaml that builds the project and runs both unit and integration tests.

version: '1.0'
  - prepare
  - test
  - build
  - 'integration test'
    title: Cloning main repository...
    stage: prepare
    type: git-clone
    repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
    revision: '${{CF_REVISION}}'
    title: Compile/Unit test
    stage: test
    image: 'maven:3.5.2-jdk-8-alpine'
      - mvn -Dmaven.repo.local=/codefresh/volume/m2_repository test      
    title: Building Docker Image
    type: build
    stage: build
    image_name: spring-boot-2-sample-app
    disable_push: true
    working_directory: ./
    tag: 'multi-stage'
    dockerfile: Dockerfile
    title: Integration test
    stage: 'integration test'
    image: maven:3.5.2-jdk-8-alpine
     - mvn -Dmaven.repo.local=/codefresh/volume/m2_repository verify
          image: '${{build_app_image}}'
            - 8080
        timeoutSeconds: 30
        periodSeconds: 15
        image: byrnedo/alpine-curl
          - "curl http://my-spring-app:8080/"

With the starting project out of the way we can define how the Git repository is created from this template in Terraform:

resource "github_repository" "my-new-java-repo" {
  name        = "my-new-java-repo"
  description = "Test repo from Terraform"
  visibility = "public"
  template {
    owner      = "kostis-codefresh"
    repository = "sample-java-project"

The snippet above instructs Terraform to create a brand new Git repository named “my-new-java-repo” that is forked/cloned from

Like Codefresh, the GitHub provider also needs a GitHub token that you can create from the GitHub Web UI.

Next we create the project and associated pipeline resource in Codefresh:

resource "codefresh_project" "example-project" {
  name = "myproject"

  tags = [

resource "codefresh_pipeline" "example-pipeline" {

  name    = "${}/build-java-app"

  tags = [

  spec {

    spec_template {
      repo        = "kostis-codefresh/${}"
      path        = "./codefresh.yml"
      revision    = "main"
      context     = "github-1"

    trigger {
      branch_regex  = "/.*/gi"
      context       = "github-1"
      description   = "Trigger for commits"
      disabled      = false
      events        = [
      modified_files_glob = ""
      name                = "commits"
      repo                = "kostis-codefresh/${}"
      type                = "git"


Notice that we also create a Git trigger for the repository that will run the pipeline in all “push” events.

It is important to note that Terraform will create all resources in order. Since inside the pipeline we use the name of the Git repo, Terraform will make sure to create the Git repository first and then the Codefresh pipeline.

To run this example you need to execute “terraform init” again in order to set up the GitHub provider.

Then once you run “terraform apply” you will see a new repository created in your Github account:

Autogenerated Github project
Autogenerated Github project

And the respective pipeline in Codefresh:

Autogenerated pipeline
Autogenerated pipeline

Notice that this pipeline is already associated with the GitHub repository that we created with Terraform. You can run this pipeline like any other Codefresh pipeline and compile/package your Java project.

Example pipeline run
Example pipeline run

If you’re keeping DRY and using shared pipelines, you can see how the above example would be modified to only create a new trigger on an already existing pipeline.

And there you have it! We have created a new project in Github and its associated without a single mouse click!

Using Codefresh to run Terraform to manage Codefresh

Creating resources with Terraform is great for automation but in the previous examples we assumed that you run the Terraform CLI manually in your terminal. You also had to inject the Codefresh API token and the GitHub Token as Terraform parameters in a manual way.

To complete the puzzle we can automate the Terraform running as well and the most natural way to do it is to use Codefresh itself. Codefresh can run Terraform on its own resulting in a strange combination where Codefresh essentially manages its own resources (via Terraform)

You can also use any of the secret facilities provided by Codefresh to store your tokens such as Kubernetes secrets, Hashicorp vault or even the built-in configuration mechanism.

As the icing on the cake, we will also create a Terraform variable that decides what is the name of the application that will be created (instead of hardcoding the value as in the previous examples)

variable "my_app_name" {
  type    = string
  default = "my-java-app"
  sensitive = false
  description = "Name of application to be created. Affects Git repo and pipeline names"

You can find the final Terraform repo at .

Now we can create the pipeline in Codefresh as any other pipeline.

Pipeline that runs terraform
Pipeline that runs terraform

Notice the two token variables on the right.
The pipeline is very simple

  1. It contains a clone step to clone the repository that contains the pipeline
  2. It runs the Terraform CLI in a freestyle step

Then we can run the pipeline and pass as parameter the application name that we want to be created:

Pass name of new application
Pass name of new application

Once the pipeline has finished, the respective Git repository and Codefresh resources will be created automatically.

Running terraform inside Codefresh
Running terraform inside Codefresh

So now we have the complete Circle of automation

  1. Codefresh is running Terraform in a “master” pipeline that is created only once
  2. Terraform is creating a Github repository
  3. Terraform is creating Codefresh resources to build the new GitHub repository
  4. A dynamic pipeline is automatically generated in Codefresh and can be used to build/compile the new project

Instead of running this pipeline yourself, you could potentially allow developers to run it on their own. This way you remove yourself completely from the picture and developers can create new applications in a completely self-service manner. Isn’t that cool?


In this blog post we have seen how you can use the Terraform provider to manage Codefresh resources. Apart from pipelines, the provider also supports other Codefresh entities such as users, teams contexts, permissions, api keys etc.

As a reminder for a production Terraform setup you should also take into account other factors such as storing correctly the shared state as well as creating a pull request workflow that shows you the plan created by Terraform before it is applied. You could even model everything in a single pipeline by using the Codefresh approval step.

In any case, you should now have some ideas on your own on how to marry Terraform and Codefresh. Happy Terraforming!

How useful was this post?

Click on a star to rate it!

Average rating 5 / 5. Vote count: 1

No votes so far! Be the first to rate this post.

Build your GitOps skills and credibility today with a GitOps Certification.

Get GitOps Certified

Ready to Get Started?
  • safer deployments
  • More frequent deployments
  • resilient deployments