Schedule a FREE onboarding and start making pipelines fast.

Pipelines-as-Code with the New Codefresh Jenkins Plugin

Containers | September 13, 2016

We are excited to announce the release of the Jenkins Codefresh plugin version 1.2.

This release brings three new important features :

  • Better branch support for Codefresh builds.
  • Ability to launch Codefresh compositions from Jenkins jobs.
  • And most exciting: support for Jenkins Pipeline.

If you’ve been using Jenkins lately, you probably heard about the Pipeline Plugin, an open-sourced rework of the workflow plugin from Cloudbees. It gives you the ability to write your whole CI/CD flow in a powerful and very comprehensible Groovy DSL.

As with all Infrastructure-as-Code (IaC) initiatives, this Pipeline-as-Code initiative by Jenkins came about to address the ever-growing scale and complexity of continuous delivery pipelines. The kind of scale where it becomes cumbersome, if not totally infeasible, to manage things manually or through the GUI, accompanied by an escalated rate of change and the need to manage multiple branches, experiments, and sandbox environments.

In a previous post, we’ve shown how you can easily complement your existing Jenkins pipelines by integrating them with Docker flows managed by Codefresh. Today we’ll be looking at even more continuous delivery goodness. We’ll start with an example of triggering Codefresh pipelines for building and testing all of the branches in your codebase.

You may want to do this if you know that some or all of your repository branches share the same dependency, and you want to retest them each time a dependency changes.

But let’s start with the basics.

Branches are Easy

In our previous post describing the Jenkins plugin we didn’t deal with branches. Branch support was added later and this is how it works:

  • If you simply choose the option to ‘Run Codefresh Build’ and there’s already a Git repository defined on your Jenkins job, the plugin will use whatever branch is used in Git definitions.
  • If you choose to build a different Codefresh service/pipeline, you’re presented with an optional ‘Branch’ field. If left empty, the default branch will be used for the build.

Screen Shot 2016-09-02 at 4.14.47 PM

Pretty simple, isn’t it?

Branches as Code

Now to the exciting stuff. Having Jenkins pipeline DSL support means we can codify the build triggers for all of our branches. This is how it can be implemented:

      1. Make sure you’re running Jenkins > 1.651.2 with the Pipeline plugin installed.
      2. If you haven’t done so yet, provide your Codefresh credentials in Jenkins system configuration as described on the plugin Wiki
      3. Create a new job by clicking ‘New Item’ on Jenkins left-hand menu.
      4. Choose ‘Pipeline’ as your job type.
      5. In the ‘Pipeline’ section of job configuration enter the Pipeline script that follows (make sure to replace the project name and git url with your service properties).
      6. Make sure to uncheck the ‘Use Groovy Sandbox’ option beneath the pipeline script window.
      7. Save your changes, run the pipeline and watch Codefresh run builds for each of the branches in your repo.

And here’s the Pipeline script:

Let’s walk through the script to see what’s going on here.

First of all, we’re defining our Git repository properties, whereas the variable service’ is also our Codefresh service name. (Note: you can create Codefresh pipelines with different names, of course. Just use a different variable).

Then, we’re calling the GitHub branches API to retrieve the list of all branches in our repo:

This brings back a list of hashmaps with branch name stored under the ‘name’ key.

We then iterate over this list, building a list of Codefresh invocations in the form:

The integration is achieved by the step name: codefresh. This gets passed the parameters:

build: true tells the plugin that we want to run a build (and not launch a composition)

cfService provides the name of the service to build

cfBranch is the name of the branch

After building the list of invocations we pass it to the special pipeline function parallel which does exactly that – executes all these Codefresh calls in parallel. Looking at the job console output we can see notification from all branches, for example, two branches: master and testing.

A few words about the ‘Groovy sandbox’ option. It’s a feature of Jenkins that allows admins to restrict script executions to a limited set of allowed methods and thus protect the system from intentional or unintentional abuse. If we leave this unchecked, our script will fail on calling groovy.json.JsonSlurper() or branchApi.newReader(). That can be solved by then going to Jenkins system config and whitelisting each of these methods, but for our use case it’s much easier to just disable sandboxing altogether. Once you decide you’re ready to use this for production, enable sandboxing and also make sure to save your pipeline script to a Jenkinsfile stored in source control. See pipeline documentation for more details.

But We Want Compositions…

Building and running unit tests in Codefresh pipelines is very important, but we usually want to bring up a composition to perform some real integration tests afterwards. Nowadays we’re mostly dealing with microservices and it’s necessary to be able to test them as a part of the whole complex system. This is the only real way to verify their communication patterns, caching abilities and overall resilience.

Codefresh provides an easy way of defining Docker compositions without leaving the service. In our example, we’ll be launching the demochat application, a fork of the open-source Slack alternative called ‘Let’s Chat’. It’s an Express.js application with a MongoDB backend. The code can be found on GitHub and here’s the Docker-Compose YAML we will use to launch it on Codefresh:

We’ll name our composition ‘mydemochat’. All the compositions on our Codefresh account are immediately visible and available for selection on Jenkins job configuration:

Screen Shot 2016-09-03 at 4.45.32 PM

This can be used in combination with build triggering or on its own. Note that if you select both build and launch options, the build will always be performed first.

Once the composition is launched, the new environment’s URL becomes available in Jenkins console output and through the Codefresh green leaves badge in the job build history.

Screen Shot 2016-09-03 at 5.31.33 PM

Compositions-As-Code

Composition launching is also available from pipeline DSL. Let’s say that in our original script — after building all the branches — we decide we want to launch a composition.

Add this to your Pipeline script:

Again we’re using the  ‘codefresh’ keyword to invoke the integration, but this time we’re passing it launch: true accompanied by the cfComposition name parameter.

If all goes well, this is what we’ll see in the job console output :

We can now browse to the composition URL and start testing our chat functionality. That’s all!

As always we’re looking forward to your feedback, and here are some links to help you get started:

Codefresh Jenkins plugin Wiki

Creating a Codefresh Composition

And if you don’t already have a Codefresh account, create a free one here.

Stay fresh!

Anton Weiss

About Anton Weiss

Ant(on) Weiss is a DevOps Evangelist and Enabler. He's been architecting software delivery processes at large and small companies for the last 15 years. Today he's the CEO and Principal Consultant at Otomato. When not evangelising for Codefresh, he's deeply involved with technical training and public speaking.

Reader Interactions

Enjoy this article? Don't forget to share.

Comments

Comments

Your email address will not be published. Required fields are marked *

Follow me on Twitter