Variables in pipelines
Codefresh provides a set of predefined variables automatically in each build, that you can use to parameterize the way your pipeline works. You can also define your own variables. Some common examples of predefined variables include:
CF_BRANCH
is the Git branch that was used for this pipeline.CF_REVISION
is the Git hash that was used for this pipeline.CF_BUILD_URL
is the url of the pipeline build.
Using Codefresh variables in your pipelines
There are two ways to use a Codefresh variable in your pipelines:
- Expose as UNIX environment variables
This is the default method, where all variables in all freestyle steps are exposed as UNIX environment variables:
$MY_VARIABLE_EXAMPLE
. - Directly in YAML properties
Use variables can be used in YAML properties with the syntax
${{MY_VARIABLE_EXAMPLE}}
.
TIP:
If you are unsure about which form you need to use, feel free to use${{MY_VARIABLE_EXAMPLE}}
everywhere. This is the Codefresh specific form and should function in all sections ofcodefresh.yml
.
Example: Print out the branch as an environment variable
In this example, we use simple echo
commands, but any program or script that reads environment variables can also read them in the same manner.
YAML
MyOwnStep:
title: Variable example
image: alpine
commands:
- echo $CF_BUILD_ID
- echo $CF_BRANCH_TAG_NORMALIZED
Example: Use directly in YAML properties
YAML
MyAppDockerImage:
title: Building Docker Image
type: build
image_name: my-own-app
tag: ${{CF_BRANCH_TAG_NORMALIZED}}
Example: Concatenating variables
YAML
MyAppDockerImage:
title: Building Docker Image
type: build
image_name: my-own-app
tag: ${{CF_BRANCH_TAG_NORMALIZED}}-${{CF_SHORT_REVISION}}
The concatenation above creates Docker images with tags such as:
master-df6a04c
develop-ba1cd68
feature-vb145dh
NOTE:
This syntax is specific to Codefresh, and is only available within the Codefresh YAML file itself. If you want to write scripts or programs that use the Codefresh variables, you need to make them aware of the environment variable form.
System variables
System variables are automatically injected to any freestyle step as environment variables.
NOTE:
It is important to understand that all Git-related variables suchCF_BRANCH
,CF_COMMIT_MESSAGE
,CF_REVISION
etc. come directly from the Git provider you use, and therefore have the same limitations of that provider.
For example, GitLab sends less information in pull request events than normal pushes, and Bitbucket sends only the short hash of a commit in pull request events. We suggest you read the documentation of your Git provider first to understand what information is available for every Git event.
Gerrit haschange-Id
andChanges
that you can map toCF_PULL_REQUEST
variables, such asCF_PULL_REQUEST_ID
and more.
Variable | Description |
---|---|
${{CF_REPO_OWNER}} |
Repository owner. |
${{CF_REPO_NAME}} |
Repository name. |
${{CF_BRANCH}} |
Branch name (or Tag depending on the payload json) of the Git repository of the main pipeline, at the time of execution. You can also use ${{CF_BRANCH_TAG_NORMALIZED}} to get the branch name normalized. It will be without any chars that are illegal in case the branch name were to be used as the Docker image tag name. You can also use ${{CF_BRANCH_TAG_NORMALIZED_LOWER_CASE}} to force lowercase. |
${{CF_BASE_BRANCH}} |
The base branch used during creation of Tag |
${{CF_PULL_REQUEST_ACTION}} |
The pull request action. Values are those defined by your Git provider such as GitHub, GitLab, Bitbucket etc. |
${{CF_PULL_REQUEST_ID}} |
The ID of the pull request. For Gerrit, use this in place of changeId . |
${{CF_PULL_REQUEST_TARGET}} |
The target branch of the pull request. For Gerrit, use this in place of Change target branch name . |
${{CF_PULL_REQUEST_NUMBER}} |
The pull request number |
${{CF_PULL_REQUEST_LABELS}} |
For GitHub and GitLab, the labels assigned to the pull requests. For Gerrit, the change hashtags . |
${{CF_PULL_REQUEST_COMMENT}} |
The comment added to the pull request. For Gerrit, use this in place of Change message . |
${{CF_PULL_REQUEST_COMMENT_AUTHOR}} |
The user who added the comment to the pull request. For Gerrit, use this in place of Change author . |
${{CF_COMMIT_AUTHOR}} |
Commit author. |
${{CF_BUILD_INITIATOR}} |
The person (username) that started the build. If the build was started by a Git webhook (e.g. from a Pull request) it will hold the webhook user. Notice that if a build is restarted manually it will always hold the username of the person that restarted it. |
${{CF_ACCOUNT}} |
Codefresh account for this build |
${{CF_COMMIT_URL}} |
Commit url. |
${{CF_COMMIT_MESSAGE}} |
Commit message of the Git repository revision, at the time of execution. The messages quotes are escaped (i.e. ‘ is not ', “ is now "). |
${{CF_COMMIT_MESSAGE_ESCAPED}} |
Commit message of the Git repository revision, at the time of execution. Special characters are escaped. |
${{CF_REVISION}} |
Revision of the Git repository of the main pipeline, at the time of execution. You can also use ${{CF_SHORT_REVISION}} to get the abbreviated 7-character revision hash, as used in Git. Note: use this variable as string with quotes to tag the image ${{CF_SHORT_REVISION}} |
${{CF_VOLUME_NAME}} |
Refers to the shared volume between freestyle steps. Normally you only need to define this in compositions. In freestyle steps, it is automatically present without any extra configuration. |
${{CF_VOLUME_PATH}} |
Refers to the mounted path of the shared volume inside a Freestyle container. In the current implementation it expands to /codefresh/volume . |
${{CF_BUILD_TRIGGER}} |
Will be an indication of the current build was triggered: build: The build was triggered from the build button webhook: The build was triggered from a control version webhook |
${{CF_BUILD_ID}} |
The build id. Note: use this variable as string with quotes to tag the image ${{CF_BUILD_ID}} |
${{CF_BUILD_TIMESTAMP}} |
The timestamp the build was created. Note: use this variable as string with quotes to tag the image ${{CF_BUILD_TIMESTAMP}} |
${{CF_BUILD_URL}} |
The URL to the build in Codefresh |
${{CF_PIPELINE_NAME}} |
The full path of the pipeline, i.e. “project/pipeline” |
${{CF_STEP_NAME}} |
the name of the step, i.e. “MyUnitTests” |
${{CF_URL}} |
The URL of Codefresh system |
${{CI}} |
The value is always true |
${{CF_KUBECONFIG_PATH}} |
Path to injected kubeconfig if at least one Kubernetes cluster is configured. You can easily run custom kubectl commands since it is automatically setup by Codefresh in all pipelines. |
Any variable specified in the pipeline settings | For example, if you configure the pipeline settings with a variable named PORT, you can put the variable in your YAML build descriptor as ${{PORT}} . |
Context-related Variables
Context-related variables are created dynamically during the workflow execution and according to the used steps.
Variable | Description |
---|---|
Working Directories | For example, you can set the working directory of step A with a variable named after a previously executed step, step B . Therefore, setting step A with working-directory:${{B}} means that step A executes in the same working directory as step B . |
Images | You can set the candidate field of the push step with a variable named after a previously executed build step. Since the details of a created image are not necessarily known ahead of time, the variable can create an association to an optionally dynamic image name. Therefore, setting push step A with candidate:${{B}} means that step A will push the image built by step B . Note that this capability works only for candidate and image fields in Codefresh steps. |
A very common pattern in Codefresh pipelines, is to create a Docker image in one step, and then run a command on its container in the next step (e.g. run unit tests):
YAML
version: '1.0'
steps:
MyAppDockerImage:
title: Building Docker Image
type: build
image_name: my-own-app
MyUnitTests:
title: Running Unit tests
image: ${{MyAppDockerImage}}
commands:
- ./my-unit-tests.sh
In the example above you can see the MyAppDockerImage
variable that denotes a Docker image created dynamically within this single pipeline. In the second step we use it as a Docker context in order to run unit tests. See also the unit testing example app.
Step variables
Every step in a Codefresh pipeline also exposes several built-in variables. You can access them via the global steps
parent variable.
- Each step creates a variable based on the name of the step. You can then use the members of each variable for status conditions such as:
steps.MyUnitTests.result == 'error'
for a step calledMyUnitTests
. - To access variables that have a non-standard (i.e. only alphanumeric and _ characters) names, use the Variable() function.
Step Member variables
Variables that are created by steps can have members. The members depend on the step type. For example if you have a build step named myBuildStep
you can get the ID of the docker image that gets created with echo ${{steps.myBuildStep.imageId}}
Step Type | Members |
---|---|
All step types |
|
Freestyle | - |
Composition | - |
Build |
|
Git-clone |
|
Push |
|
Approval |
|
GitHub release variables
GitHub allows you to create releases for marking specific Git tags for general availability.
You can set a trigger for GitHub releases. When a GitHub release happens, the following variables are also available:
Variable | Description |
---|---|
${{CF_RELEASE_NAME}} |
GitHub release title |
${{CF_RELEASE_TAG}} |
Git tag version |
${{CF_RELEASE_ID}} |
Internal ID for this release |
${{CF_PRERELEASE_FLAG}} |
true if the release if marked as non-production ready, false if it is ready for production |
GitHub Pull Request variables
When a pull request is closed in GitHub, the following variables are also available
Variable | Description |
---|---|
${{CF_PULL_REQUEST_MERGED}} |
true if the pull request was merged to base branch |
${{CF_PULL_REQUEST_HEAD_BRANCH}} |
the head branch of the PR (the branch that we want to merge to master) |
${{CF_PULL_REQUEST_MERGED_COMMIT_SHA}} |
the commit SHA on the base branch after the pull request was merged (in most cases it will be master) |
${{CF_PULL_REQUEST_HEAD_COMMIT_SHA}} |
the commit SHA on the head branch (the branch that we want to push) |
Gerrit changeId & change message variables
Gerrit has no explicit concept of pull requests, as in other version control systems, to map trigger event payloads to builds. Instead, Gerrit uses Changes
which serves a similar purpose and functionality as pull requests. You can achieve the same functionality in Codefresh with our CF_PULL_REQUEST
group of environment variables.
Variable | Description |
---|---|
${{CF_PULL_REQUEST_ID}} |
The change identified by the change-Id . |
${{CF_PULL_REQUEST_TARGET}} |
The target branch of the pull request. For Gerrit, use this in place of Change target branch name . |
${{CF_PULL_REQUEST_COMMENT}} |
The comment added to the pull request. For Gerrit, use this in place of Change message . |
${{CF_PULL_REQUEST_COMMENT_AUTHOR}} |
The user who added the comment to the pull request. For Gerrit, use this in place of Change author . |
User-defined variables
User variables can be defined at 6 levels:
- Manually within a step using the export command or in any subsequent step with the cf_export command
- Freestyle step definition (using the
environment
field) - Specific build Execution (after clicking the “Build” button open the “Build Variables” section, or use the CLI)
- Pipeline Definition (under “Environment variables” section in the pipeline view)
- Shared Configuration (defined under your account settings, and used using the “Import from shared configuration” button under the “Environment Variables” section in the pipeline view)
- Variables defined on the Project level (Under the variables tab on any project view)
The options are listed in order of priority (from the most important to the least important), so in case of multiple variables defined at different locations with the same name, the order of overriding will be as listed here.
For example if a pipeline variable is defined both in project level and as an execution parameter of a specific build, then the final result will be the value defined as a build parameter and the project level variable will not take effect.
Exporting environment variables from a freestyle step
Steps defined within steps are scoped to the step they were created in (even if you used the export
command). To use variables across steps, we provide a shared file through which you can import and export variables.
There are two ways to add variables to the shared file:
- Using the
cf_export
command - Writing directly to the file
Exporting variables with cf_export
Within every freestyle step, the cf_export
command allows you to export variables across steps by writing to the shared variables file.
The variables exported through
cf_export
overrides those at the pipeline-level.
You can either:
- Explicitly state a VAR=VAL pair
- State the name of an existing exported environment variable (like EXISTING_VAR).
version: '1.0'
steps:
freestyle-step-1:
description: Freestyle step..
title: Free styling
image: alpine:latest
commands:
# Normal export will only work in a single step
- export EXISTING_VAR=www.example.com
# CF export will now work in all other subsequent steps
- cf_export VAR1=alpine:latest VAR2=VALUE2 EXISTING_VAR
freestyle-step-2:
description: Freestyle step..
title: Free styling 2
image: ${{VAR1}}
commands:
- echo $VAR2
- echo http://$EXISTING_VAR/index.php
cf_export
syntax
cf_export
has the same syntax as the bash export command.
This means that when you use it you don’t need any dollar signs for the variable that is created/assigned.
cf_export $MY_VAR # Don't do this
cf_export MY_VAR # Correct syntax
Masking variables within cf_export
Mask variables within cf_export
by defining the --mask
flag.
Values of masked variables in cf_export
commands are replaced with asterisks in the build logs. This helps to ensure that sensitive information is not exposed also in the variables list, in addition to the logs.
Here is an example with standard and masked versions of the same variable in cf_export
commands.
version: '1.0'
steps:
freestyle-step:
description: Freestyle step..
title: Free styling
image: alpine:latest
commands:
- export EXISTING_VAR=some-value
- cf_export VAR1=alpine:latest VAR2=VALUE2 EXISTING_VAR --mask
Export variables to all steps with cf_export
By default, cf_export
works only on subsequent steps.
To export a variable both in the current and to all the remaining steps, do the following:
export MY_VAR='example' # Makes MY_VAR available in this step only
cf_export MY_VAR='example' # Makes MY_VAR available also to all steps after this one
There is nothing really magic about cf_export
. It is a normal script. You can see its contents on your own by entering the command cat /codefresh/volume/cf_export
on any Codefresh freestyle step inside a pipeline.
For more information on its limitations, see the troubleshooting page.
Directly writing to the file
For more advanced use cases, you can write directly to the shared variable file that Codefresh reads to understand which variables need to be available to all steps.
This file has a simple format where each line is a variable and its value in the form of VARIABLE=VALUE
. The cf_export
command mentioned in the previous section is just a shorthand for writing on this file.
The variables file is available inside freestyle steps in the following path: ${{CF_VOLUME_PATH}}/env_vars_to_export
version: '1.0'
steps:
freestyle-step-1:
description: Freestyle step..
title: Free styling
image: alpine:latest
commands:
- echo VAR1=192.168.0.1 >> ${{CF_VOLUME_PATH}}/env_vars_to_export
- echo hey=alpine:3.9 >> ${{CF_VOLUME_PATH}}/env_vars_to_export
freestyle-step-2:
description: Freestyle step..
title: Free styling 2
image: ${{hey}}
commands:
- echo http://$VAR1/index.php
Use this technique if you have complex expressions that have issues with the cf_export
command.
Masking variables in logs
Codefresh has the built-in capability to automatically mask variables in logs if they are encrypted. The values of encrypted variables will be replaced with asterisks in build logs.
The variables can be defined in any of the usual ways Codefresh offers such as shared configuration or within the pipeline:
Notice that this feature is currently available only in Enterprise accounts.
Encrypt variables for pipeline build runs
When you run a pipeline manually, you can either add variables on the fly or change the value of existing variables for the specific run of the build. You can also encrypt the build-specific variables, new or existing, at the same time.
- Do one of the following:
- From the Pipelines page, select the pipeline and click Run on the right.
- From the Builds page, select the pipeline to run manually, and then click Run.
- Expand Build Variables.
- To add a variable, click Add Variable.
- Enter the Key and Value for the new variable.
- To encrypt, click
Encrypt, and confirm.
Escape special characters in variables
When passing special characters through environmental variables, use \
as an escape character.
For example, to pass a cassandra connection string, you might do something like Points\=hostname\;Port\=16376\;Username\=user\;Password\=password
.
This will safely escape ;
and =
.