How to define Codefresh pipelines in a declarative manner
Codefresh offers its own built-in format for creating pipelines. The pipeline specification is based on the YAML syntax allowing you to describe your pipelines in a completely declarative manner.
Using Codefresh yaml is the recommended way to create pipelines.
Simple example for codefresh.yml
Here is a very minimal example:
It contains two steps, one named build_image that creates a docker image, and another one called perform_tests that runs unit test with
If you want to know more about how steps work in Codefresh make sure to read the introduction to Pipelines first, before moving on.
Basic pipeline syntax
You can customize your build environment (pipeline) by using the Codefresh YAML file,
codefresh.yml. Codefresh uses the build specifications in the
codefresh.yml file to execute your build. The
codefresh.yml can be basic or it can include intricate build specifications.
A YAML file is comprised of a series of steps that are executed in the order in which they are specified.
You must define a step type for each step, unless you are using a freestyle step. Each step uses Docker images and containers as facilitators for execution. For example, the Freestyle step spins up a container and executes the specified shell commands from the YAML file.
The step names should be unique within the same pipeline. This mainly affects the visualization of the pipeline when it runs.
Each step produces a resource, which you can reference in other steps, and are executed in real-time. For example, a Freestyle step can reference an image that was produced by a Build step. This allows you to chain steps together and create highly-customized builds.
Steps chaining and referencing is possible due to implementation of variables in yml file - read more on relevant section
|Freestyle||Executes one or more shell commands in a container similar to
|Build||Builds a Docker image like
|Push||Pushes a Docker image to an external registry similar to
|Git Clone||Overrides the default git clone behavior.|
|Composition||Starts a Docker Composition like
|Launch Composition||Starts a long term Docker composition that stays up after the end of the pipeline.|
|Deploy||Deploys to Kubernetes clusters.|
|Approval||Pauses a pipeline and waits for human intervention.|
For more information on creating your own step, see the steps page.
If you are editing Codefresh yaml within the Codefresh GUI, the editor will automatically highlight errors as they happen.
This allows you to make quick edits (and possibly run some builds) straight from the GUI. Once you are happy with your pipeline you should commit it to your repository as
codefresh.yml (pipeline as code).
For more information on where the YAML file can be stored see the creating pipelines page.
By default, Codefresh will execute all steps in the yaml file and instantly fail the build, if any step
presents an error. To change this behavior add the
fail_fast:false property in any step that you wish to be ignored
in case of errors.
For example, if you have a freestyle step that runs integration tests, and you don’t want the whole pipeline
to fail if any of the tests fail, add the
fail_fast line to that step:
Now the pipeline will continue to run even if the step
Notice also that by default Codefresh pipelines run in sequential mode. All steps will be executed one after
the other and in the same order as included in the
If you wish to use parallel steps in your pipelines, see the parallel steps page.
In the context of a step, a working directory can be of the following type:
|Empty||Defaults to the Codefresh volume (found at
|Variable that contains the ID of a Git-Clone step||Runs the step within the cloned directory. If you name your clone step as
|Variable that contains the ID of any other step||Runs the step within the same working directory that the specified was executed. This option is not available for for Git-Clone steps.|
|Absolute filesystem path||Treated as is within the container.|
|Relative filesystem path||Treated as relative path from the cloned directory of the service|
|‘IMAGE_WORK_DIR’||When using a freestyle step, by default the working directory will be the cloned directory of the service. Use this value in-order to use the image working directory for example:
Retrying a step
Sometimes you want to retry a step that has a problem. Network hiccups, transient failures and flaky test environments are common problems that prevent pipelines from working in a predictable manner.
Codefresh allows you to retry any of your steps with the built-in syntax:
retry: block has the following parameters:
maxAttemptsdefines how many times this step will run again if there are execution errors (default is 1 and the Max. is 10).
delayis the number of seconds to wait before each attempt (default is 5 seconds and the Max. is 60 seconds).
exponentialFactordefines how many times the delay should be multiplied by itself after each attempt (default is 1 and Max. is 5).
All parameters are optional. The exponentialFactor works like this:
- exponentialFactor=1, delay=5 => each time wait 5 seconds before trying again, no matter the number of attempts.
- exponentialFactor=2, delay=5 => first retry will have a delay of 25 seconds, third will have 125 and so on.
Here is a full example:
Notice that Codefresh also provides the following variables that allow you change your script/applications according to the retry attempts:
CF_CURRENT_ATTEMPTcontains the number of current retry attempt.
CF_MAX_ATTEMPTScontains all the number of total attempts defined.
The retry mechanism is available for all kinds of steps.
If you want to use strings inside your pipeline that create conflicts with the Codefresh syntax parser (for example they are YAML themselves) you need
to escape them using multi-line strings with the
The following pipeline is not parsed correctly because the echo command is using the yaml
You can fix this issue by using a multi-line YAML string:
|- character keeps the line breaks of the text (but removes the last one). Use the
>- character if you want to convert line breaks to spaces.
For more information see the YAML specification.
Using YAML anchors to avoid repetition
Codefresh also supports yaml anchors, references and extends. These allow you to keep your pipeline DRY.
For example, let’s say that you have two freestyle steps:
- The first one fills a MySQL server with data.
- The second one runs integration tests that use the MySQL server.
Here is the respective pipeline:
Instead of repeating the same environment variables in both steps, we can create them once and then just reference them in the second step with the
You also define anchors at the top of the pipeline in the special
Finally. you also extend steps like below:
Here we deploy to two kubernetes clusters. The first step defines the staging deployment. For the second step, we extend the first one and only change the name of the cluster to point to production. Everything else (i.e. namespace and service) are exactly the same.