Improve this page on GitHub

Sharing data between pipeline steps

How to cache folders between steps and builds

Codefresh creates a shared volume in each pipeline that is automatically shared on all freestyle steps.

Codefresh volume

All steps share the same volume

This volume exists at /codefresh/volume by default. You can simply copy files there to have them available in all Codefresh steps (as well as subsequent builds of the same pipeline)

Notice that the git clone step will delete any files not mentioned in .gitignore. Therefore if you want to cache a folder that exists in your project directory (such as node_modules) you also need to add it to .gitignore

Using the shared volume

You can see the example project at https://github.com/codefreshdemo/cf-example-shared-volumes-between-builds. The repository contains a simple application, a Dockerfile and an example pipeline that saves/reads a dummy file to the Codefresh volume

Here is the whole pipeline:

codefresh.yml

version: "1.0"
stages:
  - "clone"
  - "build"
  - "shared-volume"

steps:
  clone:
    title: "Cloning repository"
    type: "git-clone"
    repo: "codefreshdemo/cf-example-shared-volumes-between-builds"
    revision: "master"
    stage: "clone"

  build_image:
    title: "Building image"
    type: "build"
    image_name: "sample-app"
    working_directory: "${{clone}}"
    tag: "demo"
    dockerfile: "Dockerfile"
    stage: "build"
  
  copy_to_shared_volume:
    title: "Copy file to shared volume"
    type: "freestyle" 
    image: alpine:3.9 
    working_directory: "${{clone}}"
    commands:
      - ls -l /codefresh/volume/
      - cp ./artifact/artifact.example /codefresh/volume/artifact.example
    stage: "shared-volume"
    
  list_shared_volume:
    title: "List shared volume files"
    type: "freestyle" 
    image: alpine:3.9 
    working_directory: "${{clone}}"
    commands:
      - pwd
      - ls -l /codefresh/volume
    stage: "shared-volume"

This pipeline does the following:

  1. Clones the source code with a Git clone step
  2. Builds a docker image using a Build step
  3. Copies the file artifact.example to the volume with a freestyle step
  4. Reads the contents of the volume in a different freestyle step

If you run the pipeline you will see the file contents in the fourth step:

Listing volume contents

Listing volume contents

If you also run the pipeline a second time, you will see the dummy file in all steps (as the volume is automatically cached for subsequent builds as well).

Caching build dependencies and Docker layers

More information about caching build dependencies can be found in the caching documentation page as well as in this blog post.