Pipelines-as-Code with the New Codefresh Jenkins Plugin

Pipelines-as-Code with the New Codefresh Jenkins Plugin

6 min read

We are excited to announce the release of the Jenkins Codefresh plugin version 1.2.

This release brings three new important features :

  • Better branch support for Codefresh builds.
  • Ability to launch Codefresh compositions from Jenkins jobs.
  • And most exciting: support for Jenkins Pipeline.

If you’ve been using Jenkins lately, you probably heard about the Pipeline Plugin, an open-sourced rework of the workflow plugin from Cloudbees. It gives you the ability to write your whole CI/CD flow in a powerful and very comprehensible Groovy DSL.

As with all Infrastructure-as-Code (IaC) initiatives, this Pipeline-as-Code initiative by Jenkins came about to address the ever-growing scale and complexity of continuous delivery pipelines. The kind of scale where it becomes cumbersome, if not totally infeasible, to manage things manually or through the GUI, accompanied by an escalated rate of change and the need to manage multiple branches, experiments, and sandbox environments.

In a previous post, we’ve shown how you can easily complement your existing Jenkins pipelines by integrating them with Docker flows managed by Codefresh. Today we’ll be looking at even more continuous delivery goodness. We’ll start with an example of triggering Codefresh pipelines for building and testing all of the branches in your codebase.

You may want to do this if you know that some or all of your repository branches share the same dependency, and you want to retest them each time a dependency changes.

But let’s start with the basics.

Branches are Easy

In our previous post describing the Jenkins plugin we didn’t deal with branches. Branch support was added later and this is how it works:

  • If you simply choose the option to ‘Run Codefresh Build’ and there’s already a Git repository defined on your Jenkins job, the plugin will use whatever branch is used in Git definitions.
  • If you choose to build a different Codefresh service/pipeline, you’re presented with an optional ‘Branch’ field. If left empty, the default branch will be used for the build.

Screen Shot 2016-09-02 at 4.14.47 PM

Pretty simple, isn’t it?

Branches as Code

Now to the exciting stuff. Having Jenkins pipeline DSL support means we can codify the build triggers for all of our branches. This is how it can be implemented:

      1. Make sure you’re running Jenkins > 1.651.2 with the Pipeline plugin installed.
      2. If you haven’t done so yet, provide your Codefresh credentials in Jenkins system configuration as described on the plugin Wiki
      3. Create a new job by clicking ‘New Item’ on Jenkins left-hand menu.
      4. Choose ‘Pipeline’ as your job type.
      5. In the ‘Pipeline’ section of job configuration enter the Pipeline script that follows (make sure to replace the project name and git url with your service properties).
      6. Make sure to uncheck the ‘Use Groovy Sandbox’ option beneath the pipeline script window.
      7. Save your changes, run the pipeline and watch Codefresh run builds for each of the branches in your repo.

And here’s the Pipeline script:

stage 'Codefresh'

def owner = 'codefreshdemo'
def service = 'demochat'
def branchApi = new URL("https://api.github.com/repos/${owner}/${service}/branches")
def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader())
cfPerBranch = [:]

for ( branch in branches ) {

       def branchName = branch.name
       cfPerBranch[branchName] = {
           node {
             println "running Codefresh build for branch ${branchName}"
             codefresh build: true, cfService: service, cfBranch: branchName
          }
       }
}
parallel cfPerBranch

Let’s walk through the script to see what’s going on here.

First of all, we’re defining our Git repository properties, whereas the variable service’ is also our Codefresh service name. (Note: you can create Codefresh pipelines with different names, of course. Just use a different variable).

Then, we’re calling the GitHub branches API to retrieve the list of all branches in our repo:

def branches = new groovy.json.JsonSlurper().parse(branchApi.newReader())

This brings back a list of hashmaps with branch name stored under the ‘name’ key.

We then iterate over this list, building a list of Codefresh invocations in the form:

codefresh build: true, cfService: service, cfBranch: branchName

The integration is achieved by the step name: codefresh. This gets passed the parameters:

build: true tells the plugin that we want to run a build (and not launch a composition)

cfService provides the name of the service to build

cfBranch is the name of the branch

After building the list of invocations we pass it to the special pipeline function parallel which does exactly that – executes all these Codefresh calls in parallel. Looking at the job console output we can see notification from all branches, for example, two branches: master and testing.

[master] running cf build for branch master
 [Pipeline] [master] codefresh
 [Pipeline] [testing] echo
 [testing] running cf build for branch testing
 [Pipeline] [testing] codefresh
 [master]
 [master] Triggering Codefresh build. Service: codefreshdemo/demochat.
 [master] 
 [testing] 
 [testing] Triggering Codefresh build. Service: codefreshdemo/demochat.
 [testing] 
 [master] Codefresh build running - https://g.codefresh.io/process/57caca9749786005005adce8
 [master]  Waiting 5 seconds...
 [testing] Codefresh build running - https://g.codefresh.io/process/57caca977ec1d4050067147b
 [testing]  Waiting 5 seconds...

A few words about the ‘Groovy sandbox’ option. It’s a feature of Jenkins that allows admins to restrict script executions to a limited set of allowed methods and thus protect the system from intentional or unintentional abuse. If we leave this unchecked, our script will fail on calling groovy.json.JsonSlurper() or branchApi.newReader(). That can be solved by then going to Jenkins system config and whitelisting each of these methods, but for our use case it’s much easier to just disable sandboxing altogether. Once you decide you’re ready to use this for production, enable sandboxing and also make sure to save your pipeline script to a Jenkinsfile stored in source control. See pipeline documentation for more details.

But We Want Compositions…

Building and running unit tests in Codefresh pipelines is very important, but we usually want to bring up a composition to perform some real integration tests afterwards. Nowadays we’re mostly dealing with microservices and it’s necessary to be able to test them as a part of the whole complex system. This is the only real way to verify their communication patterns, caching abilities and overall resilience.

Codefresh provides an easy way of defining Docker compositions without leaving the service. In our example, we’ll be launching the demochat application, a fork of the open-source Slack alternative called ‘Let’s Chat’. It’s an Express.js application with a MongoDB backend. The code can be found on GitHub and here’s the Docker-Compose YAML we will use to launch it on Codefresh:

version: '2'
services:
 demochat:
   image: codefreshdemo/demochat:master'
   ports:
     - 5000
     - 5222
   links:
     - mongo
 mongo:
   image: mongo

We’ll name our composition ‘mydemochat’. All the compositions on our Codefresh account are immediately visible and available for selection on Jenkins job configuration:

Screen Shot 2016-09-03 at 4.45.32 PM

This can be used in combination with build triggering or on its own. Note that if you select both build and launch options, the build will always be performed first.

Once the composition is launched, the new environment’s URL becomes available in Jenkins console output and through the Codefresh green leaves badge in the job build history.

Screen Shot 2016-09-03 at 5.31.33 PM

Compositions-As-Code

Composition launching is also available from pipeline DSL. Let’s say that in our original script — after building all the branches — we decide we want to launch a composition.

Add this to your Pipeline script:

stage 'Launch'
node {
  codefresh launch: true, cfComposition: 'mydemochat'
}

Again we’re using the  ‘codefresh’ keyword to invoke the integration, but this time we’re passing it launch: true accompanied by the cfComposition name parameter.

If all goes well, this is what we’ll see in the job console output :

Launching Codefresh composition environment: demochat.
  Waiting 5 seconds...
  Codefresh environment launched successfully - http://cf-aue1-docker-node-0010.cf-cd.com:32773
  [Pipeline] }

We can now browse to the composition URL and start testing our chat functionality. That’s all!

As always we’re looking forward to your feedback, and here are some links to help you get started:

Codefresh Jenkins plugin Wiki

Creating a Codefresh Composition

And if you don’t already have a Codefresh account, create a free one here.

Stay fresh!

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

Build your GitOps skills and credibility today with a GitOps Certification.

Get GitOps Certified

Ready to Get Started?
  • safer deployments
  • More frequent deployments
  • resilient deployments