Composition

The composition step runs a Docker Composition as a means to execute finite commands in a more complex interaction of services.

Motivation for Compositions

The primary purpose of compositions is to run integration tests or any kind of tests that require multiple services for their execution.

The syntax offered by Codefresh closely follows the syntax for Docker-compose files (version 2), so if already know how Docker compose works, you will be immediately familiar with Codefresh compositions.

The big difference between the two, is that Codefresh is distinguishing between two kinds of services

  • Composition Services
  • Composition Candidates

Composition services are helper services that are needed for the tests to run. These can be a database, a queue, a cache, or the back-end docker image of your application.

Composition candidates are special services that will execute the tests. Codefresh will monitor their execution and fail the build if they do not succeed. Almost always composition candidates are Docker images that contain unit/integration tests or other kinds of tests (e.g. performance)

You need at least one composition service and one candidate for the composition step.

Composition step syntax

Here is an example of a composition. There is one composition service (PostgreSQL database) and one candidate (tests executed with gulp)

The most important part is the command line that executes the tests. If it fails, then the whole composition step will fail.

YAML

step_name:
  type: composition
  title: Step Title
  description: Free text description
  working_directory: ${{a_clone_step}}
  composition:
    version: '2'
    services:
      db:
        image: postgres
  composition_candidates:
    test_service:
      image: ${{build_step}}
      command: gulp integration_test
      working_dir: /app
      environment:
        - key=value
  composition_variables:
    - key=value
  fail_fast: false
  when:
    condition:
      all:
        notFeatureBranch: 'match("${{CF_BRANCH}}", "/FB-/", true) == false'
  on_success:
    ...
  on_fail:
    ...
  on_finish:
    ...
  retry:
    ...  
Field Description Required/Optional/Default
title The free-text display name of the step. Optional
description A basic, free-text description of the step. Optional
stage Parent group of this step. See using stages for more information. Optional
working_directory The directory in which to search for the composition file. It can be an explicit path in the container’s file system, or a variable that references another step.
The default is ${{main_clone}}.
Default
composition The composition you want to run. It can be an inline YAML definition, a path to a composition file on the file system, or the logical name of a composition stored in the Codefresh system. We support most features of Docker compose version 2.0 Required
composition_candidates The definition of the service to monitor. Each candidate has a single command parameter that decides what will be tested. Required
environment environment that will be accessible to the container Optional
composition_variables A set of environment variables to substitute in the composition. Notice that these veriables are docker-compose variables and NOT envirvonment variables Optional
fail_fast If a step fails, and the process is halted. The default value is true. Default
when Define a set of conditions which need to be satisfied in order to execute this step.
You can find more information in the Conditional Execution of Steps article.
Optional
on_success, on_fail and on_finish Define operations to perform upon step completion using a set of predefined Post-Step Operations. Optional
retry Define retry behavior as described in Retrying a step. Optional

Composition versus Composition Candidates

For Codefresh to determine if the step and operations were successfully executed, you must specify at least one composition_candidate.

A composition_candidate is a single service component of the normal Docker composition that is monitored for a successful exit code, and determines the outcome of the step. During runtime, the composition_candidate is merged into the specified composition, and is monitored for successful execution.

The critical part of each candidate is the command parameter. This takes a single command that will be executed inside the Docker container of the candidate and will decide if the whole composition is successful or not. Only one command is allowed (similar to Docker compose). If you wish to test multiple commands you need to connect them with &&.

If the composition already contains a service with the same name as the composition_candidate, the two service definitions are combined, with preference given to the composition_candidate’s definition.

For example, we create a new Codefresh composition named ‘test_composition’:

test-composition.yml

version: '2'
  services:
    db:
      image: postgres
    test_service:
      image: myuser/mytestservice:latest
      command: gulp integration_test

Now we want to reuse this composition during our build for testing purposes. We can add the following composition step to our codefresh.yml file, and define the composition step so that test_service always uses the latest image that was built.

YAML

run_tests:
  type: composition
  composition: test_composition
  composition_candidates:
    test_service:
      image: ${{build_step}}

In the above example, both composition and composition_candidates define a service named test_service. After merging these definitions, test_service will maintain the command that was defined in the original composition, but will refer to the image built by the step named build_step.

Composition variables versus environment variables

Docker compose supports two kinds of variables in its syntax.

  • There are environment variables that are used in the docker-compose file itself (${VAR} syntax)
  • There are environment variables that are passed in containers (environment: yaml group)

Codefresh supports both kinds, but notice that variables mentioned in the composition_variables yaml group refer to the first kind. Any variables defined there are NOT passed automatically to containers (use the environment yaml group for that purpose).

This can be illustrated with the following example:

codefresh.yml

version: '1.0'
steps:
  comp1:
    type: composition
    title: Composition example 1
    description: Free text description
    composition:
      version: '2'
      services:
        db:
          image: alpine
    composition_candidates:
      test_service:
        image: alpine
        command: printenv
        environment:
          - FIRST_KEY=VALUE
    composition_variables:
      - ANOTHER_KEY=ANOTHER_VALUE

If you run the composition you will see that the printenv command shows the following:

test_service_1  | FIRST_KEY=VALUE

The FIRST_KEY variable which is defined explicitly in the environment yaml part is correctly passed to the alpine container. The ANOTHER_KEY is not visible in the container at all.

You should use the composition_variables yaml group for variables that you wish to reuse in other parts of your composition using the ${ANOTHER_KEY} syntax.