Hybrid GitOps Runtime installation

Provision Hybrid GitOps Runtimes

ATTENTION:
We have transitioned to a Helm-based installation for Hybrid GitOps Runtimes for improved experience and performance, which is now the default for GitOps Runtimes.
The CLI-based installation for Hybrid GitOps is now considered legacy.
We will deprecate this installation mode permanently in the coming months. Please stay tuned for further updates and instructions, including the migration process.

Provision one or more Hybrid GitOps Runtimes in your Codefresh account.
Start by reviewing system requirements for Hybrid GitOps.
If you are installing with ingress-controllers, you must configure them as required before starting the installation.

To provision a Hosted GitOps Runtime, see Provision a hosted runtime in Set up a hosted (Hosted GitOps) environment.

Git providers and Hybrid Runtimes
Your Codefresh account is always linked to a specific Git provider. This is the Git provider you select on installing the first GitOps Runtime, either Hybrid or Hosted, in your Codefresh account. All the Hybrid GitOps Runtimes you install in the same account use the same Git provider.
If Bitbucker Server is your Git provider, you must also select the specific server instance to associate with the runtime.

To change the Git provider for your Codefresh account after installation, contact Codefresh support.

If you want to skip validating the scopes for the provided token, you must create the repositories for the runtime and for the Git Source before starting the installation.

Codefresh and Argo CD
The Hybrid GitOps Runtime comprises Argo CD components and Codefresh-specific components.

Codefresh users rely on our platform to deliver software reliably, and predictably without interruption.
To maintain that high standard, we add several weeks of testing and bug fixes to new versions of Argo before making them available within Codefresh.
Typically, new versions of Argo are available within 30 days of release in Argo.

There are two parts to installing a Hybrid GitOps Runtime:

  1. Installing the GitOps CLI
  2. Installing the Hybrid GitOps Runtime, either through the CLI wizard or via silent installation through the installation flags.
    The Hybrid GitOps Runtime is installed in a specific namespace on your cluster. You can install more Hybrid GitOps Runtimes on different clusters in your deployment.
    Every Hybrid GitOps Runtime installation makes commits to three Git repos:
    • Runtime install repo: The installation repo that manages the Hybrid Runtime itself with Argo CD. If the repo URL does not exist, it is automatically created during installation.
    • Git Source repo: Created automatically during Runtime installation. The repo where you store manifests for pipelines and applications. See Git Sources.
    • Shared configuration repo: Created for the first GitOps Runtime installed in your account. The repo stores configuration manifests for account-level resources and is shared with other GitOps Runtimes in the same account. See Shared configuration repository.

Minimum system requirements

Item Requirement
Kubernetes cluster Server version 1.18 and higher, without Argo Project components.
Tip: To check the server version, run:
kubectl version --short.
Ingress controller Configured on Kubernetes cluster and exposed from the cluster.
Supported and tested ingress controllers include:
Node requirements
  • Memory: 5000 MB
  • CPU: 2
Cluster permissions Cluster admin permissions
Git providers
  • GitHub
  • GitHub Enterprise
  • GitLab Cloud
  • GitLab Server
  • Bitbucket Cloud
  • Bitbucket Server
Git access tokens Git runtime token:
  • Valid expiration date
  • Scopes:
    • GitHub and GitHub Enterprise: repo, admin-repo.hook
    • GitLab Cloud and GitLab Server: api, read_repository
    • Bitbucket Cloud and Server: Permissions: Read, Workspace membership: Read, Webhooks: Read and write, Repositories: Write, Admin

Ingress controller configuration

You need to configure ingress controllers only if your runtime uses an ingress controller. Codefresh offers tunnel-based runtimes which do not require ingress controllers. See Access Mode in Runtime flags and Tunnel-based runtime flags.

Ambassador ingress configuration

For detailed configuration information, see the Ambassador ingress controller documentation.

This section lists the specific configuration requirements for Codefresh to be completed before installing the hybrid runtime.

  • Valid external IP address
  • Valid TLS certificate
  • TCP support

Valid external IP address

Run kubectl get svc -A to get a list of services and verify that the EXTERNAL-IP column for your ingress controller shows a valid hostname.

Valid TLS certificate

For secure runtime installation, the ingress controller must have a valid TLS certificate.

Use the FQDN (Fully Qualified Domain Name) of the ingress controller for the TLS certificate.


TCP support

Configure the ingress controller to handle TCP requests.



AWS ALB ingress configuration

For detailed configuration information, see the ALB AWS ingress controller documentation.

This table lists the specific configuration requirements for Codefresh.

What to configure When to configure
Valid external IP address Before installing hybrid runtime
Valid TLS certificate  
TCP support  
Controller configuration]  
Alias DNS record in route53 to load balancer After installing hybrid runtime
(Optional) Git integration registration  

Valid external IP address

Run kubectl get svc -A to get a list of services and verify that the EXTERNAL-IP column for your ingress controller shows a valid hostname.


Valid TLS certificate

For secure runtime installation, the ingress controller must have a valid TLS certificate.

Use the FQDN (Fully Qualified Domain Name) of the ingress controller for the TLS certificate.


TCP support

Configure the ingress controller to handle TCP requests.


Controller configuration

In the ingress resource file, verify that spec.controller is configured as ingress.k8s.aws/alb.

apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: alb
spec:
  controller: ingress.k8s.aws/alb

Create an alias to load balancer in route53

The alias must be configured after installing the hybrid runtime.

  1. Make sure a DNS record is available in the correct hosted zone.
  2. After hybrid runtime installation, in Amazon Route 53, create an alias to route traffic to the load balancer that is automatically created during the installation:
    • Record name: Enter the same record name used in the installation.
    • Toggle Alias to ON.
    • From the Route traffic to list, select Alias to Application and Classic Load Balancer.
    • From the list of Regions, select the region. For example, US East.
    • From the list of load balancers, select the load balancer that was created during installation.

For more information, see Creating records by using the Amazon Route 53 console.

Route 53 record settings for AWS ALB

Route 53 record settings for AWS ALB

(Optional) Git integration registration

If the installation failed, as can happen if the DNS record was not created within the timeframe, manually create and register Git integrations using these commands:
cf integration git add default --runtime <RUNTIME-NAME> --api-url <API-URL>
cf integration git register default --runtime <RUNTIME-NAME> --token <RUNTIME-AUTHENTICATION-TOKEN>



Istio ingress configuration

For detailed configuration information, see Istio ingress controller documentation.

The table below lists the specific configuration requirements for Codefresh.

What to configure When to configure
Valid external IP address Before installing hybrid runtime
Valid TLS certificate  
TCP support  
Cluster routing service After installing hybrid runtime

Valid external IP address

Run kubectl get svc -A to get a list of services and verify that the EXTERNAL-IP column for your ingress controller shows a valid hostname.


Valid TLS certificate

For secure runtime installation, the ingress controller must have a valid TLS certificate.

Use the FQDN (Fully Qualified Domain Name) of the ingress controller for the TLS certificate.


TCP support

Configure the ingress controller to handle TCP requests.


Cluster routing service

The cluster routing service must be configured after installing the hybrid runtime.

Based on the runtime version, you need to configure a single or multiple VirtualService resources for the app-proxy, webhook, and workflow services.

Runtime version 0.0.543 or higher

Configure a single VirtualService resource to route traffic to the app-proxy, webhook, and workflow services, as in the example below.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  namespace: pov-codefresh-istio-runtime # replace with your runtime name
  name: internal-router
spec:
  hosts:
    -  pov-codefresh-istio-runtime.sales-dev.codefresh.io   # replace with your host name
  gateways:
    - istio-system/internal-router  # replace with your gateway name
  http:
    - match:
      - uri:
          prefix: /webhooks
      route:
      - destination:
          host: internal-router
          port:
            number: 80
    - match:
      - uri:
          prefix: /app-proxy
      route:
      - destination:
          host: internal-router
          port:
            number: 80
    - match:
      - uri:
          prefix: /workflows
      route:
      - destination:
          host: internal-router
          port:
            number: 80
Runtime version 0.0.542 or lower

Configure two different VirtualService resources, one to route traffic to the app-proxy, and the second to route traffic to the webhook services, as in the examples below.


VirtualService example for app-proxy:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  namespace: test-runtime3 # replace with your runtime name
  name: cap-app-proxy 
spec:
  hosts:
    - my.support.cf-cd.com # replace with your host name
  gateways:
    - my-gateway # replace with your host name
  http:
    - match:
      - uri:
          prefix: /app-proxy 
      route:
      - destination:
          host: cap-app-proxy 
          port:
            number: 3017

VirtualService example for webhook:

Configure a uri.prefix and destination.host for each event-source if you have more than one.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  namespace: test-runtime3 # replace with your runtime name
  name: csdp-default-git-source
spec:
  hosts:
    - my.support.cf-cd.com # replace with your host name
  gateways:
    - my-gateway # replace with your gateway name
  http:
    - match:
      - uri:
          prefix: /webhooks/test-runtime3/push-github # replace `test-runtime3` with your runtime name, and `push-github` with the name of your event source
      route:
      - destination:
          host: push-github-eventsource-svc # replace `push-github' with the name of your event source
          port:
            number: 80
    - match:
      - uri:
          prefix: /webhooks/test-runtime3/cypress-docker-images-push # replace `test-runtime3` with your runtime name, and `cypress-docker-images-push` with the name of your event source
      route:
      - destination:
          host: cypress-docker-images-push-eventsource-svc # replace `cypress-docker-images-push` with the name of your event source
          port:
            number: 80


NGINX Enterprise ingress configuration

For detailed configuration information, see NGINX ingress controller documentation.

The table below lists the specific configuration requirements for Codefresh.

What to configure When to configure
Verify valid external IP address Before installing hybrid runtime
Valid TLS certificate  
TCP support  
NGINX Ingress: Enable report status to cluster  
NGINX Ingress Operator: Enable report status to cluster  
Patch certificate secret After installing hybrid runtime

Valid external IP address

Run kubectl get svc -A to get a list of services and verify that the EXTERNAL-IP column for your ingress controller shows a valid hostname.


Valid TLS certificate

For secure runtime installation, the ingress controller must have a valid TLS certificate.

Use the FQDN (Fully Qualified Domain Name) of the ingress controller for the TLS certificate.


TCP support

Configure the ingress controller to handle TCP requests.


NGINX Ingress: Enable report status to cluster

If the ingress controller is not configured to report its status to the cluster, Argo’s health check reports the health status as “progressing” resulting in a timeout error during installation.

  • Pass --report-ingress-status to deployment.
spec:                                                                                                                                                                 
  containers: 
    - args:                                                                                                                                              
      - --report-ingress-status

NGINX Ingress Operator: Enable report status to cluster

If the ingress controller is not configured to report its status to the cluster, Argo’s health check reports the health status as “progressing” resulting in a timeout error during installation.

  1. Add this to the Nginxingresscontrollers resource file:

    ...
    spec:
      reportIngressStatus:
        enable: true
    ...
    
  2. Make sure you have a certificate secret in the same namespace as the runtime. Copy an existing secret if you don’t have one.
    You will need to add this to the ingress-master when you have completed runtime installation.


Patch certificate secret

The certificate secret must be configured after installing the hybrid runtime.

Patch the certificate secret in spec.tls of the ingress-master resource.
The secret must be in the same namespace as the runtime.

  1. Go to the runtime namespace with the NGINX ingress controller.
  2. In ingress-master, add to spec.tls:

     tls:                                                                                                                                                                    
      - hosts:                                                                                                                                                                
      - <host_name>                                                                                             
      secretName: <secret_name>
    


NGINX Community version ingress configuration

Codefresh has been tested with and supports implementations of the major providers. For your convenience, we have provided configuration instructions, both for supported and untested providers in Provider-specific configuration.

This section lists the specific configuration requirements for Codefresh to be completed before installing the hybrid runtime.

  • Verify valid external IP address
  • Valid TLS certificate
  • TCP support

Valid external IP address

Run kubectl get svc -A to get a list of services, and verify that the EXTERNAL-IP column for your ingress controller shows a valid hostname.


Valid TLS certificate

For secure runtime installation, the ingress controller must have a valid TLS certificate.

Use the FQDN (Fully Qualified Domain Name) of the ingress controller for the TLS certificate.


TCP support

Configure the ingress controller to handle TCP requests.

Here’s an example of TCP configuration for NGINX Community on AWS.
Verify that the ingress-nginx-controller service manifest has either of the following annotations:

service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "tcp"
OR
service.beta.kubernetes.io/aws-load-balancer-type: nlb


Provider-specific configuration

The instructions are valid for k8s.io/ingress-nginx, the community version of NGINX.

AWS
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/aws/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
For additional configuration options, see ingress-nginx documentation for AWS.
Azure (AKS)
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
For additional configuration options, see ingress-nginx documentation for AKS.
Bare Metal Clusters
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/baremetal/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
Bare-metal clusters often have additional considerations. See Bare-metal ingress-nginx considerations.
Digital Ocean
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/do/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
For additional configuration options, see ingress-nginx documentation for Digital Ocean.
Docker Desktop
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
For additional configuration options, see ingress-nginx documentation for Docker Desktop.
Note: By default, Docker Desktop services will provision with localhost as their external address. Triggers in delivery pipelines cannot reach this instance unless they originate from the same machine where Docker Desktop is being used.
Exoscale
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/exoscale/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
For additional configuration options, see ingress-nginx documentation for Exoscale.
Google (GKE)
Add firewall rules
GKE by default limits outbound requests from nodes. For the runtime to communicate with the control-plane in Codefresh, add a firewall-specific rule.
  1. Find your cluster's network:
    gcloud container clusters describe [CLUSTER_NAME] --format=get"(network)"
  2. Get the Cluster IPV4 CIDR:
    gcloud container clusters describe [CLUSTER_NAME] --format=get"(clusterIpv4Cidr)"
  3. Replace the `[CLUSTER_NAME]`, `[NETWORK]`, and `[CLUSTER_IPV4_CIDR]`, with the relevant values:
    gcloud compute firewall-rules create "[CLUSTER_NAME]-to-all-vms-on-network"
    --network="[NETWORK]" \
    --source-ranges="[CLUSTER_IPV4_CIDR]" \
    --allow=tcp,udp,icmp,esp,ah,sctp

Use ingress-nginx
  1. Create a `cluster-admin` role binding:
    kubectl create clusterrolebinding cluster-admin-binding \
    --clusterrole cluster-admin \
    --user $(gcloud config get-value account)
  2. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml
  3. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
We recommend reviewing the provider-specific documentation for GKE.
MicroK8s
  1. Install using Microk8s addon system:
    microk8s enable ingress
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
MicroK8s has not been tested with Codefresh, and may require additional configuration. For details, see Ingress addon documentation.
MiniKube
  1. Install using MiniKube addon system:
    minikube addons enable ingress
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
MiniKube has not been tested with Codefresh, and may require additional configuration. For details, see Ingress addon documentation.
Oracle Cloud Infrastructure
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
For additional configuration options, see ingress-nginx documentation for Oracle Cloud.
Scaleway
  1. Apply:
    kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/scw/deploy.yaml
  2. Verify a valid external address exists:
    kubectl get svc ingress-nginx-controller -n ingress-nginx
For additional configuration options, see ingress-nginx documentation for Scaleway.


Traefik ingress configuration

For detailed configuration information, see Traefik ingress controller documentation.

The table below lists the specific configuration requirements for Codefresh.

What to configure When to configure
Valid external IP address Before installing hybrid runtime
Valid SSL certificate  
TCP support  
Enable report status to cluster  

Valid external IP address

Run kubectl get svc -A to get a list of services and verify that the EXTERNAL-IP column for your ingress controller shows a valid hostname.


Valid TLS certificate

For secure runtime installation, the ingress controller must have a valid TLS certificate.

Use the FQDN (Fully Qualified Domain Name) of the ingress controller for the TLS certificate.


TCP support

Configure the ingress controller to handle TCP requests.


Enable report health status to cluster

By default, the Traefik ingress controller is not configured to report its health status to the cluster or ingress resource. Argo CD is therefore unable to assess its health status as the status.loadBalancer.ingress list does not have a value for hostname or IP. During installation, ArgoCD reports the health status of the ingress controller as progressing, resulting in a timeout error.

To prevent the timeout error, add publishedService to your ingress resource configuration . This parameter populates the health status of the ingress resource, enabling Argo CD to report the correct health status during installation. For details, see Traefik Kubernetes Ingress Documentation.

Based on your Traefik ingress controller installation, update your ingress resource by adding publishedService to providers.kubernetesIngress.ingressEndpoint.

The value must be in the format "<namespace>/<service-name>"
where:
<service-name> is the Traefik service from which to copy the status

...
providers:
  kubernetesIngress:
    ingressEndpoint:
      publishedService: "<namespace>/<traefik-service>" # Example, "codefresh/traefik-default" 
...

GitOps CLI installation

GitOps CLI installation modes

The table lists the modes available to install the Codefresh CLI.

Install mode OS Commands
curl MacOS-x64 curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-darwin-amd64.tar.gz | tar zx && mv ./cf-darwin-amd64 /usr/local/bin/cf && cf version
  MacOS-m1 curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-darwin-arm64.tar.gz | tar zx && mv ./cf-darwin-arm64 /usr/local/bin/cf && cf version
  Linux - X64 curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-linux-amd64.tar.gz | tar zx && mv ./cf-linux-amd64 /usr/local/bin/cf && cf version
  Linux - ARM curl -L --output - https://github.com/codefresh-io/cli-v2/releases/latest/download/cf-linux-arm64.tar.gz | tar zx && mv ./cf-linux-arm64 /usr/local/bin/cf && cf version
brew N/A brew tap codefresh-io/cli && brew install cf2

Install the GitOps CLI

Install the GitOps CLI using the option that best suits you: curl, brew, or standard download.
If you are not sure which OS to select for curl, simply select one, and Codefresh automatically identifies and selects the right OS for the installation.

  1. Do one of the following:
    • For first-time installation, go to the Welcome page, select + Install Runtime.
    • If you have provisioned a GitOps Runtime, in the Codefresh UI, go to GitOps Runtimes, and select + Add Runtime.
  2. Install the Codefresh CLI:
    • Select one of the installation modes.
    • Generate the API key.
    • Create the authentication context: cf config create-context codefresh --api-key <generatedKey>

Download CLI to install runtime

Download CLI to install runtime


Install Hybrid GitOps Runtime

Before you begin


How to

  1. Do one of the following:
    • If this is your first Hybrid GitOps installation, in the Welcome page, select + Install Runtime.
    • If you have already provisioned a Hybrid GitOps Runtime, to provision additional runtimes, in the Codefresh UI:
      On the toolbar, click the Settings icon, and expand Runtimes in the sidebar, and select GitOps Runtimes.
  2. Click + Add Runtimes, and then select Hybrid Runtimes.
  3. Do one of the following:
    • CLI wizard: Run cf runtime install, and follow the prompts to enter the required values.
    • Silent install: Pass the required flags in the install command:
      cf runtime install <runtime-name> --repo <git-repo> --git-token <git-token> --silent
      For the list of flags, see Hybrid runtime installation flags.
  4. If relevant, complete the configuration for these ingress controllers:
  5. If you bypassed installing ingress resources with the --skip-ingress flag for ingress controllers not in the supported list, create and register Git integrations using these commands:
    cf integration git add default --runtime <RUNTIME-NAME> --api-url <API-URL>
    cf integration git register default --runtime <RUNTIME-NAME> --token <RUNTIME-AUTHENTICATION-TOKEN>

Hybrid GitOps Runtime installation flags

This section describes the required and optional flags to install a Hybrid GitOps Runtime. For documentation purposes, the flags are grouped into:

  • Runtime flags, relating to runtime, cluster, and namespace requirements
  • Ingress-less flags, for tunnel-based installation
  • Ingress-controller flags, for ingress-based installation
  • Git provider and repo flags
  • Codefresh resource flags

Runtime flags

Runtime name
Required.
The Runtime name must start with a lower-case character, and can include up to 62 lower-case characters and numbers.

  • CLI wizard: Add when prompted.
  • Silent install: Add the --runtime flag and define the name.

Namespace resource labels
Optional.
The label of the namespace resource to which you are installing the Hybrid Runtime. Labels are required to identify the networks that need access during installation, as is the case when using services meshes, such as Istio for example.

  • CLI wizard and Silent install: Add the --namespace-labels flag, and define the labels in key=value format. Separate multiple labels with commas.

Kube context
Required.
The cluster defined as the default for kubectl. If you have more than one Kube context, the current context is selected by default.

  • CLI wizard: Select the Kube context from the list displayed.
  • Silent install: Explicitly specify the Kube context with the --context flag.

Access mode
The access mode for the runtime, which can be one of the following:

  • Tunnel-based, for runtimes without ingress controllers. This is the default.
  • Ingress-based, for runtimes with ingress controllers.

  • CLI wizard: Select the Codefresh tunnel-based or Ingress-based access mode from the list displayed. Tunnel-based mode is selected by default.
  • Silent install:

Shared configuration repository
The Git repository per Runtime account with shared configuration manifests.

  • CLI wizard and Silent install: Add the --shared-config-repo flag and define the path to the shared repo.

Tunnel-based runtime flags

These flags are required to install tunnel-based Hybrid Runtimes, without an ingress controller.

IP allowlist

Optional.

The allowed list of IPs from which to forward requests to the internal customer cluster for ingress-less runtime installations. The allowlist can include IPv4 and IPv6 addresses, with/without subnet and subnet masks. Multiple IPs must be separated by commas.

When omitted, all incoming requests are authenticated regardless of the IPs from which they originated.

  • CLI wizard and Silent install: Add the --ips-allow-list flag, followed by the IP address, or list of comma-separated IPs to define more than one. For example, --ips-allow-list 77.126.94.70/16,192.168.0.0

Ingress controller flags

Skip ingress
Required, if you are using an unsupported ingress controller.
For unsupported ingress controllers, bypass installing ingress resources with the --skip-ingress flag.
In this case, after completing the installation, manually configure the cluster’s routing service, and create and register Git integrations. See the last step in Install the Hybrid GitOps Runtime.

Ingress class
Required.

  • CLI wizard: Select the ingress class for Runtime installation from the list displayed.
  • Silent install: Explicitly specify the ingress class through the --ingress-class flag. Otherwise, Runtime installation fails.

Ingress host
Required.
The IP address or host name of the ingress controller component.

  • CLI wizard: Automatically selects and displays the host, either from the cluster or the ingress controller associated with the Ingress class.
  • Silent install: Add the --ingress-host flag. If a value is not provided, takes the host from the ingress controller associated with the Ingress class.

    Important: For AWS ALB, the ingress host is created post-installation. However, when prompted, add the domain name you will create in Route 53 as the ingress host.

Insecure ingress hosts
TLS certificates for the ingress host:
If the ingress host does not have a valid TLS certificate, you can continue with the installation in insecure mode, which disables certificate validation.

  • CLI wizard: Automatically detects and prompts you to confirm continuing the installation in insecure mode.
  • Silent install: To continue with the installation in insecure mode, add the --insecure-ingress-host flag.

Internal ingress host
Optional.
Enforce separation between internal (app-proxy) and external (webhook) communication by adding an internal ingress host for the app-proxy service in the internal network.
For both CLI wizard and Silent install:


Git provider and repo flags

The Git provider defined for the Runtime.

Because Codefresh creates a shared configuration repo for the Runtimes in your account, the Git provider defined for the first Runtime you install in your account is used for all the other Runtimes in the same account.

You can define any of the following Git providers:


Skip token scopes validation

Optional.
Skip validating scopes for the token provided (for any Git provider). This flag can be useful for GitHub with fine-grained tokens, as these are currently (March 23) still in Beta according to GitHub, and therefore not offically supported by Codefresh. The tokens should work if they have the correct scopes.

To skip token validation, add --skip-permission-validation true.

IMPORTANT:
Before using this flag, review the required scopes for runtime tokens.

When defined, Codefresh does not validate the scopes assigned to the token provided. If the token does not include the scopes required for Codefresh to automatically create the repositories for the runtime and Git Source during installation, the installation will fail.
The alternative is to create both repos before the installation.

GitHub

GitHub is the default Git provider for Hybrid Runtimes. Being the default provider, for both the CLI wizard and Silent install, you need to provide only the repository URL and the Git runtime token.

For the required scopes, see GitHub and GitHub Enterprise Runtime token scopes.

--repo <repo_url> --git-token <git-runtime-token>

where:

  • --repo <repo_url> (required), is the HTTPS clone URL of the Git repository for the Runtime installation, including the .git suffix. Copy the clone URL from your GitHub website (see Cloning with HTTPS URLs).
    If the repo doesn’t exist, copy an existing clone URL and change the name of the repo. Codefresh creates the repository during the installation.

    Repo URL format:
    https://github.com/<owner>/reponame>.git[/subdirectory][?ref=branch]
    where:

    • <owner>/<reponame> is your username or organization name, followed by the name of the repo, identical to the HTTPS clone URL. For example, https://github.com/nr-codefresh/codefresh.io.git.
    • [/subdirectory] (optional) is the path to a subdirectory within the repo. When omitted, the Runtime is installed in the root of the repository. For example, /runtimes/defs.
    • [?ref=branch] (optional) is the ref queryParam to select a specific branch. When omitted, the Runtime is installed in the default branch. For example, codefresh-prod.

    Example:
    https://github.com/nr-codefresh/codefresh.io.git/runtimes/defs?ref=codefresh-prod

  • --git-token <git-runtime-token> (required), is the Git token authenticating access to the Runtime installation repository (see GitHub runtime token scopes).


GitHub Enterprise

For the required scopes, see GitHub and GitHub Enterprise runtime token scopes.

--provider github --repo <repo_url> --git-token <git-runtime-token>

where:

  • --provider github (required), defines GitHub Enterprise as the Git provider for the Runtime and the account.
  • --repo <repo_url> (required), is the HTTPS clone URL of the Git repository for the Runtime installation, including the .git suffix. Copy the clone URL for HTTPS from your GitHub Enterprise website (see Cloning with HTTPS URLs).
    If the repo doesn’t exist, copy an existing clone URL and change the name of the repo. Codefresh creates the repository during the installation. Repo URL format:

    https://ghe-trial.devops.cf-cd.com/<owner>/<https-repo-url>.git[/subdirectory][?ref=branch]
    where:

    • <owner>/<reponame> is your username or organization name, followed by the name of the repo. For example, codefresh-io/codefresh.io.git.
    • [/subdirectory] (optional) is the path to a subdirectory within the repo. When omitted, the Runtime is installed in the root of the repository. For example, /runtimes/defs.
    • [?ref=branch] (optional) is the ref queryParam to select a specific branch. When omitted, the Runtime is installed in the default branch. For example, codefresh-prod.

    Example:
    https://ghe-trial.devops.cf-cd.com/codefresh-io/codefresh.io.git/runtimes/defs?ref=codefresh-prod

  • --git-token <git-runtime-token> (required), is the Git token authenticating access to the Runtime installation repository (see GitHub runtime token scopes).

GitLab Cloud

For the required scopes, see GitLab Cloud and GitLab Server runtime token scopes.

--provider gitlab --repo <https_project_url> --git-token <git_runtime_token>

where:

  • --provider gitlab (required), defines GitLab Cloud as the Git provider for the Runtime and the account.
  • --repo <repo_url> (required), is the HTTPS clone URL of the Git project for the Runtime installation, including the .git suffix. Copy the clone URL for HTTPS from your GitLab website.
    If the repo doesn’t exist, copy an existing clone URL and change the name of the repo. Codefresh creates the repository during the installation.

    Important: You must create the group with access to the project prior to the installation.

    Repo URL format:

    https://gitlab.com/<owner>/<project_name>.git[/subdirectory][?ref=branch]
    where:

    • <owner> is either your username, or if your project is within a group, the front-slash separated path to the project. For example, nr-codefresh (owner), or parent-group/child-group (group hierarchy)
    • <projectname> is the name of the project. For example, codefresh.
    • [/subdirectory] (optional) is the path to a subdirectory within the repo. When omitted, the Runtime is installed in the root of the repository. For example, /runtimes/defs.
    • [?ref=branch] (optional) is the ref queryParam to select a specific branch. When omitted, the Runtime is installed in the default branch. For example, codefresh-prod.

    Examples:
    https://gitlab.com/nr-codefresh/codefresh.git/runtimes/defs?ref=codefresh-prod (owner)

    https://gitlab.com/parent-group/child-group/codefresh.git/runtimes/defs?ref=codefresh-prod (group hierarchy)

  • --git-token <git-runtime-token> (required), is the Git token authenticating access to the Runtime installation repository (see GitLab runtime token scopes).

GitLab Server

For the required scopes, see GitLab Cloud and GitLab Server runtime token scopes.

--provider gitlab --repo <https_project_url> --git-token <git_runtime_token>

where:

  • --provider gitlab (required), defines GitLab Server as the Git provider for the Runtime and the account.
  • --repo <repo_url> (required), is the HTTPS clone URL of the Git repository for the Runtime installation, including the .git suffix.
    If the project doesn’t exist, copy an existing clone URL and change the name of the project. Codefresh creates the project during the installation.

    Important: You must create the group with access to the project prior to the installation.

    Repo URL format:
    https://gitlab-onprem.devops.cf-cd.com/<owner>/<project_name>.git[/subdirectory][?ref=branch]
    where:

    • <owner> is your username, or if the project is within a group or groups, the name of the group. For example, nr-codefresh (owner), or parent-group/child-group (group hierarchy)
    • <projectname> is the name of the project. For example, codefresh.
    • [/subdirectory] (optional) is the path to a subdirectory within the repo. When omitted, the Runtime is installed in the root of the repository. For example, /runtimes/defs.
    • [?ref=branch] (optional) is the ref queryParam to select a specific branch. When omitted, the Runtime is installed in the default branch. For example, codefresh-prod.

    Examples:
    https://gitlab-onprem.devops.cf-cd.com/nr-codefresh/codefresh.git/runtimes/defs?ref=codefresh-prod (owner)

    https://gitlab-onprem.devops.cf-cd.com/parent-group/child-group/codefresh.git/runtimes/defs?ref=codefresh-prod (group hierarchy)

  • --git-token <git-runtime-token> (required), is the Git token authenticating access to the Runtime installation repository (see GitLab runtime token scopes).

Bitbucket Cloud

For the required scopes, see Bitbucket runtime token scopes.

--provider bitbucket --repo <https_repo_url> --git-user <git_username> --git-token <git_runtime_token>

where:

  • --provider gitlab (required), defines Bitbucket Cloud as the Git provider for the Runtime and the account.
  • --repo <repo_url> (required), is the HTTPS clone URL of the Git repository for the Runtime installation, including the .git suffix.
    If the project doesn’t exist, copy an existing clone URL and change the name of the project. Codefresh creates the project during Runtime installation.

    Important: Remove the username, including @ from the copied URL.

    Repo URL format:

    https://bitbucket.org<workspace_id><https-repo-url>.git[/subdirectory][?ref=branch]
    where:

    • <workspace_id> is your workspace ID. For example, nr-codefresh.
    • <repo_name> is the name of the repository. For example, codefresh.
    • [/subdirectory] (optional) is the path to a subdirectory within the repo. When omitted, the Runtime is installed in the root of the repository. For example, /runtimes/defs.
    • [?ref=branch] (optional) is the ref queryParam to select a specific branch. When omitted, the Runtime is installed in the default branch. For example, codefresh-prod.

    Example:
    https://bitbucket.org/nr-codefresh/codefresh.git/runtimes/defs?ref=codefresh-prod

  • --git-user <git_username> (required), is your username for the Bitbucket Cloud account.
  • --git-token <git-runtime-token> (required), is the Git token authenticating access to the runtime installation repository (see Bitbucket runtime token scopes).

Bitbucket Server

For the required scopes, see Bitbucket runtime token scopes.

--provider bitbucket-server --repo <repo_url> --git-user <git_username> --git-token <git_runtime_token>

where:

  • --provider gitlab (required), defines Bitbucket Cloud as the Git provider for the Runtime and the account.
  • --repo <repo_url> (required), is the HTTPS clone URL of the Git repository for the Runtime installation, including the .git suffix.
    If the project doesn’t exist, copy an existing clone URL and change the name of the project. Codefresh then creates the project during the installation.

    Important: Remove the username, including @ from the copied URL.

    Repo URL format:

    https://bitbucket-server-8.2.devops.cf-cd.com:7990/scm/<owner_org_name>/<repo_name>.git[/subdirectory][?ref=branch]
    where:

    • <owner_org_name> is your username or organization name. For example, codefresh-io..
    • <repo_name> is the name of the repo. For example, codefresh.
    • [/subdirectory] (optional) is the path to a subdirectory within the repo. When omitted, the Runtime is installed in the root of the repository. For example, /runtimes/defs.
    • [?ref=branch] (optional) is the ref queryParam to select a specific branch. When omitted, the Runtime is installed in the default branch. For example, codefresh-prod.

    Example:
    https://bitbucket-server-8.2.devops.cf-cd.com:7990/scm/codefresh-io/codefresh.git/runtimes/defs?ref=codefresh-prod

  • --git-user <git_username> (required), is your username for the Bitbucket Server account.
  • --git-token <git-runtime-token> (required), is the Git token authenticating access to the Runtime installation repository (see Bitbucket runtime token scopes).


Codefresh resource flags

Codefresh demo resources
Optional.
Install demo pipelines to use as a starting point to create your own GitOps pipelines. We recommend installing the demo resources as these are used in our quick start tutorials.

  • Silent install: Add the --demo-resources flag, and define its value as true (default), or false. For example, --demo-resources=true

Insecure flag
For on-premises installations, if the Ingress controller does not have a valid SSL certificate, to continue with the installation, add the --insecure flag to the installation command.



(Optional) Internal ingress host configuration for existing Hybrid GitOps Runtimes

If you already have provisioned Hybrid GitOps Runtimes, to use an internal ingress host for app-proxy communication and an external ingress host to handle webhooks, change the specs for the Ingress and Runtime resources in the Runtime installation repository. Use the examples as guidelines.

<runtime-install-repo>/apps/app-proxy/overlays/<runtime-name>/ingress.yaml: change host

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: codefresh-cap-app-proxy
  namespace: codefresh #replace with your runtime name
spec:
  ingressClassName: nginx
  rules:
  - host: my-internal-ingress-host # replace with the internal ingress host for app-proxy
    http:
      paths:
      - backend:
          service:
            name: cap-app-proxy 
            port:
              number: 3017
        path: /app-proxy/
        pathType: Prefix

../<runtime-install-repo>/bootstrap/<runtime-name>.yaml: add internalIngressHost

apiVersion: v1
data:
  base-url: https://g.codefresh.io
  runtime: |
    apiVersion: codefresh.io/v1alpha1
    kind: Runtime
    metadata:
      creationTimestamp: null
      name: codefresh #replace with your runtime name
      namespace: codefresh #replace with your runtime name
    spec:
      bootstrapSpecifier: github.com/codefresh-io/cli-v2/manifests/argo-cd
      cluster: https://7DD8390300DCEFDAF87DC5C587EC388C.gr7.us-east-1.eks.amazonaws.com
      components:
      - isInternal: false
        name: events
        type: kustomize
        url: github.com/codefresh-io/cli-v2/manifests/argo-events
        wait: true
      - isInternal: false
        name: rollouts
        type: kustomize
        url: github.com/codefresh-io/cli-v2/manifests/argo-rollouts
        wait: false
      - isInternal: false
        name: workflows
        type: kustomize
        url: github.com/codefresh-io/cli-v2/manifests/argo-workflows
        wait: false
      - isInternal: false
        name: app-proxy
        type: kustomize
        url: github.com/codefresh-io/cli-v2/manifests/app-proxy
        wait: false
      defVersion: 1.0.1
      ingressClassName: nginx
      ingressController: k8s.io/ingress-nginx
      ingressHost: https://support.cf.com/
      internalIngressHost: https://my-internal-ingress-host # add this line and replace my-internal-ingress-host with your internal ingress host
      repo: https://github.com/NimRegev/my-codefresh.git
      version: 99.99.99

Managing external clusters in GitOps Runtimes
Monitoring & managing GitOps Runtimes
Managing Git Sources in GitOps Runtimes
Shared Configuration Repository
Troubleshoot Hybrid Runtime installation