Git-clone step

Check out code in your pipelines

Clones a Git repository to the filesystem.

A pipeline can have any number of Git clone steps (even none). You can check out code from any private or public repository. Cloning a repository is not constrained to the trigger of a pipeline. You can trigger a pipeline from a commit that happened on Git repository A while the pipeline is checking out code from Git Repository B.

Usage

YAML

step_name:
  type: git-clone
  title: Step Title
  description: Step description
  working_directory: /path
  repo: owner/repo
  git: my-git-provider
  revision: abcdef12345'
  use_proxy: false
  credentials:
    username: user
    password: credentials
  fail_fast: false
  strict_fail_fast: true
  when:
    branch:
      ignore: [ develop ]
  on_success:
    ...
  on_fail:
    ...
  on_finish:
    ...
  retry:
    ...  

Fields

Exported resources:

• Working Directory

NOTE
If you want to extend the git-clone step you can use the freestyle step. Example how to do it you can find here.

Basic clone step (project-based pipeline)

The easiest way to use a Git clone step is to use your default Git provider as configured in built-in Git integrations.

Here is an example of a pipeline that will automatically check out the repository that triggered it (i.e. a commit happened on that repository).

NOTE
The name of the clone step is main_clone. This will automatically set the working directory of all other steps that follow it inside the folder of the project that was checked out. This only applies to built-in Codefresh steps and not custom plugins. This is normally what you want for a pipeline that only checks out a single project. If you use any other name apart from main_clone the working directory for all subsequent steps will not be affected and it will default on the shared volume which is the parent folder of checkouts.

codefresh.yml

version: '1.0'
steps:
    main_clone:
        title: 'Cloning main repository...'
        type: git-clone
        repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
        revision: '${{CF_REVISION}}'
        git: my-git-provider
    PrintFileList:
        title: 'Listing files'
        image: alpine:latest
        commands:
            - 'ls -l'

The CF values will be automatically filled by Codefresh from the Git trigger. See Variables in pipelines for more details.

Choosing a specific Git provider (project-based pipeline)

If you don’t want to use the default Git provider, you can explicitly set the provider by using the same name as in the integration shown in the Git integrations page.

Here is an example for an integration with the GitLab provider already connected:

codefresh.yml

version: '1.0'
steps:
    main_clone:
        title: 'Cloning main repository...'
        type: git-clone
        repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
        revision: '${{CF_REVISION}}'
        git: my-gitlab
    PrintFileList:
        title: 'Listing files'
        image: alpine:latest
        commands:
            - 'ls -l'

Check out a specific repository/revision (project based pipeline)

If you want to check out a specific git repository regardless of what repository actually created the trigger, you can just define all values in a non-static manner. For example, if you want your pipeline to always check out Git repository foo even when the trigger happened from repository bar you can define the checkout step as below:

codefresh.yml

version: '1.0'
steps:
    main_clone:
        title: 'Cloning main repository...'
        type: git-clone
        repo: 'my-github-username/foo'
        revision: '${{CF_REVISION}}'
        git: my-github-integration
    PrintFileList:
        title: 'Listing files'
        image: alpine:latest
        commands:
            - 'ls -l'

In a similar manner you can also define that the pipeline will always check out master, regardless of the commit that actually triggered it.

codefresh.yml

version: '1.0'
steps:
    main_clone:
        title: 'Cloning main repository...'
        type: git-clone
        repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
        revision: 'master'
        git: my-git-provider
    PrintFileList:
        title: 'Listing files'
        image: alpine:latest
        commands:
            - 'ls -l'

Check out code using the Codefresh Runner

If you are using the Codefresh runner, you need to use the fully qualified path of the Git repository:

codefresh.yml

version: '1.0'
steps:
    main_clone:
        title: 'Cloning main repository...'
        type: git-clone
        repo: https://github-internal.example.com/my-username/my-app
        revision: '${{CF_REVISION}}'
        git: my-internal-git-provider
    PrintFileList:
        title: 'Listing files'
        image: alpine:latest
        commands:
            - 'ls -l'

For more details, see Checking out code from a private Git repository.

Check out multiple Git repositories

It is very easy to check out additional repositories in a single pipeline by adding more git-clone steps. In that case you should use different names for the steps (instead of main_clone) as this will make the working folder for all steps the shared volume.

codefresh.yml

version: '1.0'
steps:
    my_first_checkout:
        title: 'Cloning first repository...'
        type: git-clone
        repo: 'my-gitlab-username/foo'
        revision: '${{CF_REVISION}}'
        git: my-gitlab-integration
    my_second_checkout:
        title: 'Cloning second repository...'
        type: git-clone
        repo: 'my-github-username/bar'
        revision: '${{CF_REVISION}}'
        git: my-github-integration
    PrintFileList:
        title: 'Listing files'
        image: alpine:latest
        commands:
            - 'ls -l'

Skip or customize default clone (repository-based pipeline)

If you have existing pipelines connected to repositories (only for Codefresh accounts created before May 2019) a git-clone step is transparently added to Git attached pipelines without you having to explicitly add a step into the pipeline. This is a convenience to enable easy CI pipelines.
If you do not require Git cloning, or you would like to customize the implicit Git cloning behavior, you can choose to skip the automatically added git-clone step.

There are 2 ways to do that:

1. Add a pipeline environment variable called CF_SKIP_MAIN_CLONE with value of true.

   -or-

2. Add a step with key main_clone to your pipeline. This step can be of any type and can do any action. This step will override the default clone implementation. For example:

version: '1.0'
steps:
  main_clone:
    title: Checking out code
    image: alpine/git:latest
    commands:
      - git clone ...
  another_step:
    ...

Add a timeout to terminate step execution

To prevent steps from running beyond a specific duration if so required, you can add the timeout flag to the step.
When defined:

• The timeout is activated at the beginning of the step, before the step pulls images.
• When the step’s execution duration exceeds the duration defined for the timeout, the step is automatically terminated.

NOTE
To define timeouts for parallel steps, see Adding timeouts for parallel steps.

Here’s an example of the timeout field in the step:

step_name:
  type: git-clone
  title: Step Title
  description: Step description
  working_directory: /path
  repo: owner/repo
  git: my-git-provider
  revision: abcdef12345'
  use_proxy: false
  timeout: 45m
  credentials:
    username: user
    password: credentials
  fail_fast: false
  when:
    branch:
      ignore: [ develop ]
  on_success:
    ...
  on_fail:
    ...
  on_finish:
    ...
  retry:
    ...

Timeout info in logs
Timeout information is displayed in the logs, as in the example below.

Reuse a Git token from Codefresh integrations

You also have the capability to use one of your existing Git integrations as an authentication mechanism.

The Codefresh CLI can read one of the connected Git authentication contexts and use that token for a custom clone step.

Here is an example for GitHub:

version: '1.0'
steps:
  get_git_token:
    title: Reading GitHub token
    image: codefresh/cli
    commands:
      - cf_export --mask GITHUB_TOKEN=$(codefresh get context github --decrypt -o yaml | yq -r .spec.data.auth.password)
  main_clone:
    title: Checking out code
    image: alpine/git:latest
    commands:
      - git clone https://my-github-username:$GITHUB_TOKEN@github.com/my-github-username/my-repo.git
  another_step:
    ...

Working with Git submodules

To check out a Git project including its submodules, you can use the Codefresh submodule plugin. This plugin is already offered as a public docker image at Docke Hub.

To use this module in your pipeline, add a new step like the one shown below.

version: '1.0'
steps:
  updateSubmodules:
    image: codefresh/cfstep-gitsubmodules
    environment:
      - GITHUB_TOKEN=<github_token>
      - CF_SUBMODULE_SYNC=<boolean to determine if modules should be synced>
      - CF_SUBMODULE_UPDATE_RECURSIVE=<boolean to determine if modules should be recursively updated>

The GitHub token can be either defined in the pipeline on its own as an environment variable, or fetched from the existing Git integration as shown in the previous section.

Here is full pipeline example:

codefresh.yml

version: '1.0'
stages:
  - checkout
  - prepare
  - build
steps:
  clone:
    title: Cloning the repository
    type: git-clone
    stage: checkout
    arguments:
      repo: '${{CF_REPO_OWNER}}/${{CF_REPO_NAME}}'
      git: github
      revision: '${{CF_REVISION}}'  

  updateSubmodules:
    image: codefresh/cfstep-gitsubmodules
    stage: prepare
    working_directory: '${{clone}}'
    environment:
      - GITHUB_TOKEN=${{MY_GITHUB_TOKEN}}  
  docker_build:
    title: Building docker image
    type: build
    stage: build
    working_directory: '${{clone}}/k8s/docker'
    tag: current
    disable_push: true
    image_name: 'my-docker-image'

This pipeline does the following:

1. Clones the main source code
2. Updates submodules
3. Creates a docker image

Use an SSH key with Git

It is also possible to use an SSH key with Git. When creating your pipeline, add your SSH key as an encrypted environment variable after processing it with tr:

cat ~/.ssh/my_ssh_key_file | tr '\n' ','

Then in the pipeline use it like this:

codefresh.yml

version: '1.0'
steps:
  main_clone:
    title: Checking out code
    image: alpine/git:latest
    commands:
      - mkdir -p ~/.ssh
      - echo "${SSH_KEY}" | tr \'"${SPLIT_CHAR}"\' '\n' > ~/.ssh/id_rsa
      - chmod 600 ~/.ssh/id_rsa
      - git clone <git@github.com>:my-github-username/my-repo.git
      # can also use go get or other similar command that uses git internally
  another_step:
    ...

Using Git behind a proxy

If you use the Codefresh Runner and need to use a network proxy in your clone step you need to set the variables HTTP_PROXY and/or HTTPS_PROXY in the pipeline and then activate the property use_proxy: true in the clone step. Example:

codefresh.yml

version: "1.0"
steps:
  clone:
    title: "Cloning repository"
    type: "git-clone"
    repo: "<https://github.com/my-github-user/my-repo/>"
    revision: "master"
    use_proxy: true
    git: my-git-provider

For setting the values of the proxy variables you can use any of the supported methods for defining variables such as shared configuration.

For more details, see the behind the firewall page.

Troubleshooting git-clone step

Access denied when cloning

1. Ensure you use the right name for the git integration in your git-clone step.
2. Go to Account Settings (gear icon) -> Pipeline Integrations -> Git.
   • If you do not have access, please contact one of your team’s admins.
3. Ensure that the git Integration allows access to all users.
4. Ensure that the account used for the git integration has the appropriate access.
   • Update the git integration with a new token.
   • This could be that the user associated with the git intergration does not have access to the repos.
5. Create a new git integration and reference it in the git-clone step.

NOTE
When deleting a git integration, you may get an error saying, “The integration cannot be deleted while other objects are using it.” When you see this error, it means an existing pipeline/trigger is referencing the integration. Please refer to How To: Find pipelines using specific Git integration on how to find which pipline is using the git integration.

Related articles

Creating pipelines
YAML steps
Custom Git Commands