ArgoCD ApplicationSet: Multi-Cluster Deployment Made Easy (with Code Examples)

Article Content

What Is ArgoCD ApplicationSet? 

Argo CD is a tool for deploying applications in a declarative manner, using Git as the source of truth for the desired state of the applications. It allows teams to manage their applications and infrastructure as code, and automatically synchronize them with a Kubernetes cluster.

ArgoCD ApplicationSet is a new feature within Argo CD, which provides a new, more scalable way to define and manage applications in Argo CD. ApplicationSet lets you deploy multiple applications across multiple clusters. It simplifies the management and makes it easier to handle a large number of applications and clusters.

Unlike with a traditional Argo CD Application resource, which deploys resources from a single Git repository to a single cluster or namespace, ApplicationSet uses a templating mechanism to create or modify multiple Argo CD applications at the same time, targeting multiple destination clusters or namespaces.

Note: From ArgoCD version 2.3, the ArgoCD ApplicationSet comes with the standard installation. If you are using an earlier version, you’ll need to install ApplicationSet.

Use Cases for ArgoCD ApplicationSet 

Deploying a Single Application to Multiple Clusters

One of the main use cases of ArgoCD ApplicationSet is deploying a single application to multiple clusters. This is particularly useful in a microservice architecture where you need to deploy the same service across different environments.

With ApplicationSet, you can define the application once and specify the different clusters where the application needs to be deployed. The ApplicationSet controller then interacts with Argo CD to ensure that the application is deployed to all the specified clusters.

Deploying Multiple Applications to a Single Cluster

Another use case for ArgoCD ApplicationSet is deploying multiple applications to a single cluster. This is often the case for standard software you want to install in every cluster such as nginx, cert-manager, prometheus, or sealed-secrets.

With ApplicationSet, you can define all the applications and deploy them to a single cluster. The ApplicationSet controller ensures that all the applications are deployed and maintained in the specified cluster.

Deploying Multiple Applications to Multiple Clusters

The third use case for ArgoCD ApplicationSet is deploying multiple applications to multiple clusters. This is a complex scenario that often arises in large organizations with numerous applications and environments.

With ApplicationSet, you can define the applications and the clusters where they need to be deployed. The ApplicationSet controller then ensures that all the applications are deployed to the correct clusters.

How the ApplicationSet Controller Works 

The ApplicationSet controller operates in parallel with Argo CD, within the same namespace. This controller automatically constructs Argo CD Applications in line with the specifications of a newly created ApplicationSet Custom Resource (CR).

The List Generator

Here is an example showing how the List generator works in the ApplicationSet controller. It uses the guestbook example application from the official Argo repo. Here, the ApplicationSet is deploying a web server across multiple clusters, facilitated by Argo CD:

apiVersion: argoproj.io/v1alpha1 kind: ApplicationSet
metadata:
  name: guestbook
spec:
  generators:
  - list: 
      # The template below uses these parameters
      elements: 
      - cluster: development
        url: https://1.2.3.4
      - cluster: production
        url: https://5.6.7.8
      - cluster: staging
        url: https://9.9.1.1
  template: 
    # inserts cluster and url values from parameters defined above
    metadata:
      name: '{{cluster}}-guestbook
    spec:
      project: default
      source:
        repoURL: https://github.com/argoproj-labs/applicationset.git
        targetRevision: HEAD
        path: examples/list-generator/guestbook/{{cluster}}
      destination:
        server: '{{url}}'
        namespace: guestbook

The ‘list’ subfield under the ‘generators’ section represents an instance of the List generator. The primary function of Generators is to construct a set of key-value pairs. These pairs are then inputted to the template as parameters in {{param}} format.

The template is basically an Argo CD Application, except it is parameterized with information like project, source Git repository, and destination cluster. It can accept parameters using the {{param}} syntax from the generator. The Generators are tasked with creating these parameters, while the template takes care of inserting them. The end result is a group of Argo CD Applications created from the processed template.

This illustrates how managing multiple Argo CD Applications can become much more convenient by managing them as a single ApplicationSet resource.

The Cluster Generator

Cluster Generator makes it possible to deploy applications to specific Kubernetes clusters you have configured and managed using ArgoCD. Secrets are used to provision this cluster configuration, and the ApplicationSet controller uses these Kubernetes secrets to generate parameters for every individual cluster.

The Cluster Generator supplies the following parameters for each cluster:

  • name: Cluster name in ArgoCD – taken from the ‘name’ field in the secret
  • server: Server URI, taken from the ‘server’ field of the secret
  • metadata.labels.*: Key-value pairs extracted from each label in the cluster secret
  • metadata.annotations.*: Key-value pairs taken from each annotation in the cluster secret

By default, the Cluster Generator targets all Kubernetes clusters managed by ArgoCD. However, you can use a label to target specific clusters. The following example shows how to do this:

apiVersion: argoproj.io/v1alpha1 
kind: ApplicationSet
metadata:
  name: webserver
spec:
  generators:
  - clusters: {}
  template:
    metadata:
      name: '{{name}}-webserver'
    spec:
      project: default
      source:
        repoURL: https://github.com/myrepo/infrastructure.git
        targetRevision: HEAD
        path: "kubernetes/resources/webserver/{{name}}"
      destination:
        name: "{{server}}"
        namespace: webserver-system
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:- CreateNamespace=true

Consider a case where you have two Kubernetes clusters. ApplicationSet will use both the ‘name’ and ‘server’ fields from the cluster Secret to compose the metadata.name, spec.source.path, and spec.destination.name.

Based on the above configuration, the ApplicationSet will spin off two separate ArgoCD Applications, one for each Kubernetes cluster. This process gives you a useful framework for deploying multi-cluster applications using ArgoCD and ApplicationSet.

The Matrix Generator

The Matrix Generator lets you combine parameters from two different generators. For example, here is how to deploy a list of applications to all your clusters, combining the user of the List and Cluster generator:

apiVersion: argoproj.io/v1alpha1 
kind: ApplicationSet
metadata:
  name: matrix-example
spec:
  generators:
    - matrix:
        generators:
          - clusters: {}
          - list:
              elements:
                - appName: application-1
                  namespace: app1-system
                - appName: application-2
                  namespace: app2-system
                - appName: application-3
                  namespace: app3-system
  template:
    metadata:
      name: "{{name}}-{{appName}}"
      annotations:
        argocd.argoproj.io/manifest-generate-paths: ".;.."
    spec:
      project: default
      source:
        repoURL: https://github.com/myrepo/infrastructure.git
        targetRevision: HEAD
        path: "resources/{{appName}}/{{name}}"
        helm:
          releaseName: "{{appName}}"
          valueFiles:
            - ../common.values.yaml
            - values.yaml
      destination:
        name: "{{name}}"
        namespace: "{{namespace}}"
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true

Here’s a breakdown of the code:

  • apiVersion, kind, metadata: These fields define the resource type (ApplicationSet) and include metadata about the resource, such as its name (matrix-example).
  • spec.generators - matrix: This specifies that the generator used is the matrix type. The matrix generator combines the output of other generators – in this case, clusters and list.
    • The clusters generator does not have any additional configuration in this snippet, which means it will select all clusters managed by Argo CD.
    • The list generator is configured with a list of elements, each specifying an appName and a namespace. These elements provide specific values for deploying applications.
  • template: This section defines the template for the Argo CD Applications that will be created. The matrix generator will create a combination of applications for each cluster and application pair derived from the generators.
  • metadata: Within the metadata, the name of each application is a combination of the cluster name, {{name}}, and the application name, {{appName}}, both of which are dynamically substituted from the generators’ output. The annotations field includes a path for the manifest generation, allowing Argo CD to find and use the correct Kubernetes manifest files.
  • spec: This section specifies the configuration for each Argo CD Application.

By combining the Cluster and List generators with a dynamic template, the matrix generator lets you create Argo CD Application resources for a combination of different applications across multiple clusters, simplifying the management of multi-cluster deployments.

Learn more in our detailed guide to Argo Kubernetes

Learn More in the Manning Report: GitOps with ArgoCD

Software teams that adopt GitOps deploy more often, have fewer regressions and recover from failures more quickly. GitOps works, as evidenced by many success stories we at Codefresh have seen first hand. Learn more about GitOps in an in-depth guide by Alexander Matyushentsev, one of the founders of the Argo project. Argo is the world’s fastest growing and most popular open-source GitOps tool in the world today. Usres love it, and that is the reason Codefresh has based its enterprise platform on the beloved tool.

Learn to practice GitOps with ArgoCD in the Manning Report: GitOps with ArgoCD

Related Argo CD articles