Caching the wha-?
Caching your build workspace means that the filesystem that is used throughout your build process is persisted once the build has completed, and is reused on any subsequent builds.
Great, but why would I need to cache?
There are quite a few benefits to caching but the most significant benefit probably comes from caching build dependencies.
Many high level languages and some operating systems use sophisticated package management systems to solve the “Dependency Hell” problem and to resolve dependency libraries from remote repositories.
Starting a new build from scratch with a clean filesystem means that any library dependency must be resolved and downloaded on each build. For small projects this may be a matter of seconds, but for larger projects this process could take up to a few hours (depending on different i/o parameters).
By caching the library dependencies and re-using the same filesystem workspace between builds, you can save a significant amount of time and network traffic.
So how does Codefresh cache my builds?
Codefresh uses Docker as a facilitator in the execution of the majority of the build steps; this means that almost every step of the build is executed as commands within a special Docker container created for that step.
On top of that, Codefresh provisions a portable filesystem (we call the “Codefresh Volume”) and auto-wires it at the same path to every build step container: /codefresh/volume
To clarify, consider the following codefresh.yml:
version: '1.0'
steps:
test_it:
description: Test the module
image: node:latest
working-directory: ${{main_clone}}
commands:
- npm install
- gulp test
Before any step is executed, Codefresh provisions a git-clone container (called “main_clone”), then clones your git repository and stores it in the root of the Codefresh Volume:
Then, to execute the test_it step, Codefresh provisions Container based on the specified image – node:latest, in our case. Codefresh then wires the same Volume used by the “main_clone” step to the container, and executes all the specified commands in the same working directory that was exposed by “main_clone” (/codefresh/volume/repo-name):
Running npm install will resolve all library dependencies and store them in /codefresh/volume/repo-name/node_modules.
Once the build completes, Codefresh will securely persist the Volume.
When we build again Codefresh will reuse the persisted Volume, only this time our build won’t take so long because:
- Our repo is already fully cloned, so any upcoming build will only perform a fetch and rebase.
- Our node_modules directory is already populated with all our dependencies, so npm install will only complete deltas, if any.
Awesome, how can I cache my own builds?
If your build tool resolves build dependencies into the project directory (such as NPM) we’ve got great news for you – your builds are already cached! Because Codefresh clones the project into the Volume and then persists it, all your project-level dependencies are maintained.
If your build tool resolves dependencies and stores them under $USER_HOME (such as Gradle or Maven) – caching is super easy!
Simply point your build tool’s home variable to the Codefresh Volume.
Using Gradle, you can add a switch the command:
gradle -g /codefresh/volume/.gradle
Or, set the environment variable:
GRADLE_USER_HOME=/codefresh/volume/.gradle
Using Maven, you can modify the localRepository attribute of your settings.xml file to point to /codefresh/volume/.m2.
And that’s all there is to it!