Trigger Codefresh pipeline with Docker image push

TL;DR

Have you ever wanted to run an automated DevOps pipeline not only for git push, but also in response to an external event?

Then you are in the right place. With Codefresh pipelines you can execute pipelines in response to external events.

The first event we are adding support to, is a Docker Hub push event.

When a new Docker image is pushed to Docker Hub we can trigger a webhook to execute CD pipelines.

Continuous Delivery Pipelines

When constructing Continuous Delivery pipelines and selecting a pipeline orchestration engine, you should consider the following things.

Find a robust orchestration engine

There are multiple engines to select from and you are can choose whatever you are familiar with, or the one that is easy to learn and extend.

But it’s critical, that the orchestration engine you select supports event-driven pipeline execution. These events should not only be “code related events”, but can also trigger CD pipeline.

Some examples of engines are:

GitHub Release
Docker Hub push
S3 bucket update
Kubernetes deployment update
Periodic/cron timer and etc.

Most of the multiple orchestration engines mentioned above, are tuned to continuous integration use cases where your pipeline runs only when new code is pushed into a code repository. However, when you want to trigger pipeline execution with something other than a git push event, it requires a lot of custom development and scripting. It’s also hard to setup and maintain it over time.

Therefore, in Codefresh, we have re-evaluated our approach to pipelines and changed our system to be easily adaptable to external events.

Codefresh container-based pipelines

Codefresh container-based pipelines allow the automation of multiple DevOps tasks, like build, test, analyze, deploy, configure, and release software service. We already support triggering pipelines based on push events with integrations into the most popular git services such as: GitHub, Bitbucket or GitLab.

As a user, you can start Codefresh pipeline execution manually or by using the Codefresh CLI tool.

To extend our support of Continuous Delivery use cases, we are adding support for external triggers starting with Docker Hub push event and adding more events such as:

Generic webhooks
Cron timers,
Object storage updates,
push from other Docker registries, and more.

We are also planning to open Codefresh triggers and allow our users to add support for events from other systems.

Triggering Codefresh Pipelines

Our first trigger allows triggering Codefresh pipeline execution when a new Docker image is pushed (or tagged) in a Docker Hub repository.

For this demo, I’m using the codefresh/fortune Docker image. It’s a small Alpine based image that contains a single “fortune” phrase (/fortune.txt file), automatically generated for each image build.

Codefresh exposes a web method endpoint for accepting Docker Hub webhook payload. It parses Docker Hub event and passes information about this event into a connected Codefresh pipeline, together with original webhook JSON payload.

Note: Currently, we support setting Codefresh pipeline execution trigger only through the API and codefresh command line tool.

Installing Codefresh CLI tool

Codefresh CLI allows you to work with Codefresh service using command line interface.

Download codefresh CLI tool from codefresh-io/cli.

On macOS, it’s possible to install codefresh CLI tool from our Homebrew tap:

$ brew tap codefresh-io/cli
$ brew install codefresh

Trigger Types

Trigger type represents new events that can trigger Codefresh pipeline execution. It can generate events (like cron) or can be connected to an external service to generate events from it. This can be done using push or pool mechanism.

To list all available trigger types, installed on Codefresh, use codefresh get trigger-types command.

$ codefresh get trigger-types -o yaml type: registry kind: dockerhub uri-template: 'registry:dockerhub:{{namespace}}:{{name}}:push' uri-regex: '^registry:dockerhub:[a-z0-9_-]+:[a-z0-9_-]+:push It's possible to see, that only one trigger type is currently available - registry/dockerhub. Let's take a look at a trigger type attributes: - type - is a trigger type name; registry (Docker registry) in our case - kind - some types may be of different "kind". For example, there are multiple Docker registries: Docker Hub, Codefresh CFCR, Amazon ECR, Google GCR and others. - uri-template - every external event has a unique identifier; trigger type can provide a template that helps to construct such identifier - uri-regex - regular expression used to validate event unique identifier

Trigger Event

Trigger event is an event generated or received by a Codefresh trigger type service. First, declare a new trigger event, you would like to receive in Codefresh.

$ codefresh create trigger-event 
    --type registry 
    --kind dockerhub 
    --secret XYZ1234 
    --value namespace=codefresh 
    --value name=fortune 
    --context dockerhub

Successfully created registry:dockerhub:codefresh:fortune:push trigger event.

Fore some trigger events, Codefresh can subscribe to these events on a remote system. For others, you can get detailed instruction how to configure a remote syste3m.

Run the command below, to get a detailed information about the previously generated trigger event.

$ codefresh get trigger-event -o yaml registry:dockerhub:codefresh:fortune:push

uri: 'registry:dockerhub:codefresh:fortune:push'
type: registry
kind: dockerhub
secret: XYZ1234
status: active
endpoint: 'https://g.codefresh.io/nomios/dockerhub?secret=XYZ1234'
description: Docker Hub codefresh/fortune push event
help: >-
Docker Hub webhooks fire when an image is built in, pushed or a new tag is
added to, your repository.

Configure Docker Hub webhooks on
https://hub.docker.com/r/codefresh/fortune/~/settings/webhooks/

Add following Codefresh Docker Hub webhook endpoint
https://g.codefresh.io/nomios/dockerhub?secret=XYZ1234

Docker Hub webhooks

Docker Hub webhooks are fired when an image is built, pushed or a new tag is
added.

To configure Docker Hub webhooks on https://hub.docker.com/r/codefresh/fortune/~/settings/webhooks/ add the following Codefresh Docker Hub webhook endpoint https://g.codefresh.io/nomios/dockerhub?secret=XYZ1234

For Docker Hub trigger events, it is possible to get detailed instructions on how to set up a Docker Hub webhook and copy endpoint URL, including an automatically generated webhook secret key.

Link Trigger Event to the Codefresh pipeline

First, select a Codefresh pipeline in you would like to run when a new Docker image is pushed or tagged in its Docker Hub repository.

# list Codefresh pipelines filtered by specific REPO-OWNER/REPO-NAME
$ codefresh get pipelines --repo-owner --repo-name
# copy pipeline ID for the pipeline you want to run ...
$ PIPELINE_ID=XXX....

To connect the selected Codefresh pipeline to the specified Docker Hub trigger event (registry:dockerhub:codefresh:fortune:push in our example) use the following command:

# link Docker Hub codefresh/fortune push trigger event to the specified pipeline
$ codefresh link registry:dockerhub:codefresh:fortune:push $PIPELINE_ID

Configure a Docker Hub Webhook

Navigate to the Docker Hub image home page and then to the `Webhooks` settings, or use a link from codefresh get trigger-event command output.
Copy trigger event endpoint URL into a Webhook URL field.

Running Codefresh Pipeline

When Codefresh runs a pipeline as a result of Docker Hub trigger event, it generates several environment variables available for all the steps in the pipeline.

The original event payload is always injected as EVENT_PAYLOAD environment variables.

Other injected variables are prefixed with EVENT_ prefix.

For Docker Hub trigger event, the following variables are available:

- EVENT_NAME - Docker image name
- EVENT_TAG - Docker image tag name
- EVENT_NAMESPACE - Docker Hub namespace
- EVENT_PUSHED_AT - Docker image push timestamp (RFC 3339 formatted)
- EVENT_PUSHER - Docker Hub user who pushed the image

Summary

That's all folks!

Now every time you push a new Docker image into Docker Hub, all connected Codefresh pipelines will be executed.

New to Codefresh? Get started with Codefresh by signing up for an account today!
It’s possible to see, that only one trigger type is currently available – registry/dockerhub.

Let’s take a look at a trigger type attributes:

– type – is a trigger type name; registry (Docker registry) in our case
– kind – some types may be of different “kind”. For example, there are multiple Docker registries: Docker Hub, Codefresh CFCR, Amazon ECR, Google GCR and others.
– uri-template – every external event has a unique identifier; trigger type can provide a template that helps to construct such identifier
– uri-regex – regular expression used to validate event unique identifier

Trigger Event

Trigger event is an event generated or received by a Codefresh trigger type service.

First, declare a new trigger event, you would like to receive in Codefresh.

Fore some trigger events, Codefresh can subscribe to these events on a remote system. For others, you can get detailed instruction how to configure a remote syste3m.

Run the command below, to get a detailed information about the previously generated trigger event.

Docker Hub webhooks

Docker Hub webhooks are fired when an image is built, pushed or a new tag is
added.

For Docker Hub trigger events, it is possible to get detailed instructions on how to set up a Docker Hub webhook and copy endpoint URL, including an automatically generated webhook secret key.

Link Trigger Event to the Codefresh pipeline

First, select a Codefresh pipeline in you would like to run when a new Docker image is pushed or tagged in its Docker Hub repository.

To connect the selected Codefresh pipeline to the specified Docker Hub trigger event (registry:dockerhub:codefresh:fortune:push in our example) use the following command:

Configure a Docker Hub Webhook

Navigate to the Docker Hub image home page and then to the `Webhooks` settings, or use a link from codefresh get trigger-event command output.
Copy trigger event endpoint URL into a Webhook URL field.

Running Codefresh Pipeline

When Codefresh runs a pipeline as a result of Docker Hub trigger event, it generates several environment variables available for all the steps in the pipeline.

The original event payload is always injected as EVENT_PAYLOAD environment variables.

Other injected variables are prefixed with EVENT_ prefix.

For Docker Hub trigger event, the following variables are available:

– EVENT_NAME – Docker image name
– EVENT_TAG – Docker image tag name
– EVENT_NAMESPACE – Docker Hub namespace
– EVENT_PUSHED_AT – Docker image push timestamp (RFC 3339 formatted)
– EVENT_PUSHER – Docker Hub user who pushed the image

Summary

That’s all folks!

Now every time you push a new Docker image into Docker Hub, all connected Codefresh pipelines will be executed.

New to Codefresh? Get started with Codefresh by signing up for an account today!