Schedule a FREE onboarding and start making pipelines fast.

Docker Continuous Integration with Jenkins and Codefresh

Containers | May 18, 2016

Many development shops have made a big investment in Jenkins for their CI, but with the growing popularity of containers such as Docker, there is a need for a new kind of CI/CD platform, one that is container-native.

That said, not all of the software components in a typical build process are whole applications that can be packaged and delivered in a Docker container. Instead, they are  external components, such as libraries, modules, drivers, etc.  For these components we actually need to use a traditional CI server.

The result is a need to combine a traditional CI server, such as Jenkins, for non-containerized components, together with a container-native CI tool for the final application images.


In this post, we will show exactly how you can connect Jenkins to the Codefresh container-native CI, resulting in a continuous integration pipeline for both utility components and containerized applications.

We’ll use an example pipeline composed of two building blocks:

  • a very simple npm package
  • an Express.js web application that uses this package

Let’s say the npm package is being developed by one of our dev teams. Whenever they push a change to the git repository,  Jenkins gets triggered to perform the following flow:

  1. Jenkins bumps the package version, runs unit tests — and if they pass — publishes the package to npm registry.
  2. Jenkins then triggers Codefresh to rebuild the container image which pulls in the latest npm package version.
  3. If the build is successful, another request to Codefresh is performed to bring up the application for testing.

The npm package we are building performs left-pad string manipulation. In light of the recent npm gate scandal, it seems like a good idea to develop this advanced functionality in-house 🙂

The source code for the package can be found here and it’s dead simple:

Now let’s look at how our Codefresh service is defined. The application source code can be found here

In the source tree there’s a Dockerfile:

FROM node:4.4.3

MAINTAINER Anton Weiss <anton@otomato.link>

LABEL Description=”Serve left-padded strings”

WORKDIR /var/www

ADD . /var/www

CMD npm install && npm start

This defines that we are installing npm packages for every image build. The dependency on the cf-left-pad package is defined in package.json like this:

 

“cf-left-pad” : “*”

 

This ensures that the latest version of the package will get pulled in for each ‘npm install’ execution.

You can see the service definition in Codefresh:

 Screen Shot 2016-05-05 at 1.02.20 PM

 

Let’s start building our pipeline!

Our first Jenkins job will be called ‘build-cf-left-pad’. We’ll configure it to pull the code from the GitHub repo and to run the following as ‘execute shell’ build step:

 

This script uses the ci-npm-publish package that allows passing npm registry credentials on CLI for npm publishing.

‘npm version patch’ bumps the package version number. This number is then saved to the version.properties file. And ‘npm-publish’ takes care of the publishing.

The following screenshot shows the build step configuration in Jenkins:

Screen Shot 2016-05-05 at 12.11.04 PM

Once the first build job is finished, it triggers another job named ‘call-codefresh’ The purpose of this job is to trigger an image rebuild on Codefresh. To do this, I used Jenkins Http Request plugin, configured like this:

Screen Shot 2016-05-05 at 12.16.50 PM

You can see I’m using https://g.codefresh.io/api/builds/${CODEFRESH_SERVICE_ID} API endpoint with my Codefresh access token for authorization.

This triggers a new image build and outputs the unique build id, which Jenkins saves to the buildid.properties file to pass it to the downstream job.

The downstream job is called ‘launch-cf-service’. It now launches the service we just built the image for using the following shell script code:

export SERVICE_SHA=curl --header "Content-Type: application/json" --header \ "Accept: application/json" --header "x-access-token: ${CF_TOKEN}"\

    "https://g.codefresh.io/api/builds" | jq ".builds[] |.[] | select(._id == "\"${CF_BUILD}\"") | .sha"

curl -X POST –header “Content-Type: application/json” –header “Accept: application/json”\

       –header “x-access-token: ${CF_TOKEN}” \

      -d “{\”repoOwner\”: \”antweiss\”,

 \”repoName\”: \”left-padder\”,

 \”sha\”: ${SERVICE_SHA},

 \”branch\”: \”master\”,

 \”repoData\”: {

   \”url\”: {

     \”https\”: \”https://github.com/antweiss/left-padder.git\”

   }

 }

}” https://g.codefresh.io/api/runtime/testit

 

That’s a bit more complicated, so let’s walk through it.

The first command finds the git commit hash of the latest service image build by sending a request to  https://g.codefresh.io/api/builds and parsing that with the wonderful jq tool to find the commit that corresponds to the ${CF_BUILD} id we got from the upstream job.

The second command sends a request to the https://g.codefresh.io/api/runtime/testit endpoint with a json that describes the service we want to launch and the git commit of the last image we want to launch it from. This launches the service on Codefresh, and with that, our short pipeline is complete.

Let’s look at it in action.

The build pipeline on Jenkins:

Screen Shot 2016-04-25 at 6.11.46 PM

Output of the first job (the npm build and publish):

Screen Shot 2016-04-25 at 6.15.15 PM

We can see the version of the published npm package is now 0.0.16.

Second job calls Codefresh service rebuild and gets back the build ID:

Screen Shot 2016-04-25 at 6.15.55 PM

On the Codefresh console we see a new build appear:

Screen Shot 2016-04-25 at 6.13.46 PM

Meanwhile Jenkins triggers the launch-cf-service job, passing it the build ID as a parameter:

Screen Shot 2016-04-25 at 6.17.05 PM

Back in Codefresh, we can launch the app, enter the parameters and test if it works:

Screen Shot 2016-04-25 at 6.14.31 PM

And if we check the container log, we will see that the application in fact uses the latest npm package version:

Screen Shot 2016-04-25 at 6.14.39 PM

Mission accomplished!

By the way, you can browse our Jenkins jobs here.

We’d love to hear your feedback about this flow , so that we can improve it.

 

Anton Weiss

About Anton Weiss

Ant(on) Weiss is a DevOps Evangelist and Enabler. He's been architecting software delivery processes at large and small companies for the last 15 years. Today he's the CEO and Principal Consultant at Otomato. When not evangelising for Codefresh, he's deeply involved with technical training and public speaking.

Reader Interactions

Enjoy this article? Don't forget to share.

Comments

Your email address will not be published. Required fields are marked *

Follow me on Twitter