Sharing data between pipeline steps
How to cache folders between steps and builds
Codefresh creates a shared volume in each pipeline that is automatically shared on all freestyle steps.
This volume exists at /codefresh/volume
by default. You can simply copy files there to have them available in all Codefresh steps (as well as subsequent builds of the same pipeline)
Notice that the git clone step will delete any files not mentioned in
.gitignore
. Therefore if you want to cache a folder that exists in your project directory (such asnode_modules
) you also need to add it to.gitignore
Using the shared volume
You can see the example project at https://github.com/codefreshdemo/cf-example-shared-volumes-between-builds. The repository contains a simple application, a Dockerfile and an example pipeline that saves/reads a dummy file to the Codefresh volume
Here is the whole pipeline:
codefresh.yml
version: "1.0"
stages:
- "clone"
- "build"
- "shared-volume"
steps:
clone:
title: "Cloning repository"
type: "git-clone"
repo: "codefreshdemo/cf-example-shared-volumes-between-builds"
revision: "master"
stage: "clone"
build_image:
title: "Building image"
type: "build"
image_name: "sample-app"
working_directory: "${{clone}}"
tag: "demo"
dockerfile: "Dockerfile"
stage: "build"
copy_to_shared_volume:
title: "Copy file to shared volume"
type: "freestyle"
image: alpine:3.9
working_directory: "${{clone}}"
commands:
- ls -l /codefresh/volume/
- cp ./artifact/artifact.example /codefresh/volume/artifact.example
stage: "shared-volume"
list_shared_volume:
title: "List shared volume files"
type: "freestyle"
image: alpine:3.9
working_directory: "${{clone}}"
commands:
- pwd
- ls -l /codefresh/volume
stage: "shared-volume"
This pipeline does the following:
- Clones the source code with a Git clone step
- Builds a docker image using a Build step
- Copies the file
artifact.example
to the volume with a freestyle step - Reads the contents of the volume in a different freestyle step
If you run the pipeline you will see the file contents in the fourth step:
If you also run the pipeline a second time, you will see the dummy file in all steps (as the volume is automatically cached for subsequent builds as well).
Caching build dependencies and Docker layers
More information about caching build dependencies can be found in the caching documentation page as well as in this blog post.