Using Argo CDs new Config Management Plugins to Build Kustomize, Helm, and More

Using Argo CDs new Config Management Plugins to Build Kustomize, Helm, and More

7 min read

Starting with Argo CD 2.4, creating config management plugins or CMPs via configmap has been deprecated, with support fully removed in Argo CD 2.8. While many folks have been using their own config management plugins to do things like `kustomize –enable-helm`, or specify specific version of Helm, etc – most of these seem to have not noticed the old way of doing things has been removed until just now!

The documentation has a handy guide for migrating plugins but it leaves it to the user to decide what kind of CMP they want to build. In this post, I’ll show you how to build a CMP to run Kustomize with Helm (a very popular pattern).

You can find this example, and hopefully others soon, in the repo todaywasawesome/argocd-cmp-plugin-examples. For this plugin, an example installation is provided for 

How CMPs work in Argo CD

The CMP system works by adding an arbitrary container sidecar to `argocd-repo-server` and passing in configuration and mounting the CMP server service. Repos are checked out using repo-server and then mounted into the CMP sidecar to allow rendering. Any CMP you create will ultimately have 1 job, render manifests that Argo CD can apply. This is very easy to set up!

Step 1 – Create a Configuration for our CMP

First, we need to create a configuration for our CMP. Our plugin will be called `kustomize-build-with-helm`. Here’s what it looks like:

apiVersion: v1
kind: ConfigMap
metadata:
  name: kustomize-build-with-helm
data:
  plugin.yaml: |
    apiVersion: argoproj.io/v1alpha1
    kind: ConfigManagementPlugin
    metadata:
      name: kustomize-build-with-helm
    spec:
      generate:
        command: [ "sh", "-c" ]
        args: [ "kustomize build --enable-helm" ]

That file looks funny, doesn’t it?

CMPs use a file that looks just like a Kubernetes CRD called `ConfigManagementPlugin` but actually, it’s just a yaml file that we need mounted in our sidecar. To do this we’re going to create a `ConfigMap` and mount it into our CMP sidecar. You can also bake this file into your sidecar image if you would like.

CMPs support a ton of additional options like:

  • Automatically discovering when the plugin should be run by using a glob expression to find files
  • Running init commands every time the CMP runs
  • Creating parameters that can be exposed and configured in the Argo CD UI
  • And lots more, check out the example file for all parameters

Our plugin is very simple by contrast, it’s only going to build manifests using `kustomize build –enable-helm` and that’s it. 

Step 2 – Create a CMP Sidecar

The sidecar has to have the same name you specified in the config file we just created. It also will need a Docker image to run that contains our tools. Installing tools during the `init` step is not recommended because it runs every time a CMP runs. For the sake of simplicity, we’re going to use `alpine/k8s` a ~220MB image that containers kubectl, helm, kustomize and a few other tools. If course you can build your own slim image but at the time of writing, no one has offered their own images for Argo CD CMP plugins. 

Here’s what our sidecar will look like:

containers:
- name: kustomize-build-with-helm
  command: [/var/run/argocd/argocd-cmp-server] # Entrypoint should be Argo CD lightweight CMP server i.e. argocd-cmp-server
  # image: busybox # This can be off-the-shelf or custom-built image
  image: alpine/k8s:1.26.8
  securityContext:
    runAsNonRoot: true
    runAsUser: 999
  volumeMounts:
    - mountPath: /var/run/argocd
      name: var-files
    - mountPath: /home/argocd/cmp-server/plugins
      name: plugins
    # Remove this volumeMount if you've chosen to bake the config file into the sidecar image.
    - mountPath: /home/argocd/cmp-server/config/plugin.yaml
      subPath: plugin.yaml
      name: kustomize-build-with-helm
    # Starting with v2.4, do NOT mount the same tmp volume as the repo-server container. The filesystem separation helps 
    # mitigate path traversal attacks.
    - mountPath: /tmp
      name: cmp-tmp
volumes:
- configMap:
    name: kustomize-build-with-helm
  name: kustomize-build-with-helm
- emptyDir: {}
  name: cmp-tmp

This container will need to be added to our `argocd-repo-server` deployment and I’ll show how to do that using Kustomize in just a moment. 

Let’s take a moment to explore this file.

We specify our image `alpine/k8s:1.26.8` – of course you’ll want this to match your Kubernetes version. If you’re supporting multiple versions of Kubernetes, then you can create an additional CMP plugin for each version! 

Then we mount several paths, first we mount two volumes that will run our Argo CD CMP service so repo-server can communicate with the sidecar. Next, we mount the `ConfigMap` we created earlier. Finally, we mount an empty directory to serve as a temp folder. It’s important that we not mount the same temp folder `argocd-repo-server` uses as that would make path traversal attacks easier. (Read more about running Argo CD securely).

Step 3 – Add our CMP to Argo CD (bonus, using Kustomize!)

To install this we need to modify our Argo CD instance. We can either edit the Kubernetes manifests directly to add the sidecar to the `argocd-repo-server` deployment, edit our Helm values, or we can add it in using Kustomize (see example repo). 

For this blog, I’m using Argo CD Autopilot which uses Kustomize to install Argo CD. 

First, I’ll create a file for my `configmap` and name it `kustomize-with-helm.configmap.yaml`.

apiVersion: v1
kind: ConfigMap
metadata:
  name: kustomize-build-with-helm
data:
  plugin.yaml: |
    apiVersion: argoproj.io/v1alpha1
    kind: ConfigManagementPlugin
    metadata:
      name: kustomize-build-with-helm
    spec:
      generate:
        command: [ "sh", "-c" ]
        args: [ "kustomize build --enable-helm" ]

Then I’ll create a file for my sidecar and name it `kustomize-with-helm-sidecar.yaml`

apiVersion: apps/v1
kind: Deployment
metadata:
  name: argocd-repo-server
spec:
  template:
    spec:
      containers:
      - name: kustomize-build-with-helm
        command: [/var/run/argocd/argocd-cmp-server] # Entrypoint should be Argo CD lightweight CMP server i.e. argocd-cmp-server
        # image: busybox # This can be off-the-shelf or custom-built image
        image: alpine/k8s:1.26.8
        securityContext:
          runAsNonRoot: true
          runAsUser: 999
        volumeMounts:
          - mountPath: /var/run/argocd
            name: var-files
          - mountPath: /home/argocd/cmp-server/plugins
            name: plugins
          # Remove this volumeMount if you've chosen to bake the config file into the sidecar image.
          - mountPath: /home/argocd/cmp-server/config/plugin.yaml
            subPath: plugin.yaml
            name: kustomize-build-with-helm
          # Starting with v2.4, do NOT mount the same tmp volume as the repo-server container. The filesystem separation helps 
          # mitigate path traversal attacks.
          - mountPath: /tmp
            name: cmp-tmp
      volumes:
      - configMap:
          name: kustomize-build-with-helm
        name: kustomize-build-with-helm
      - emptyDir: {}
        name: cmp-tmp

Finally, I’ll update my `kustomization.yaml` to create the new configmap and patch in my sidecar. I’ve removed some config from this file for the sake of brevity. If you’re using Argo CD Autopilot it will also have a configmap generator to create the git credentials.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: argocd
resources:
- github.com/argoproj-labs/argocd-autopilot/manifests/base?ref=v0.4.15
- kustomize-with-helm.yaml # Adds our configmap
patches:
- path: kustomize-with-helm.sidecar.yaml # Adds the sidecar to argocd-repo-server

Now we can commit that, or run a `kubectl apply -k .` on the Kustomization we just created.

Step 4 – Configure Applications to use our Plugin

Finally, we didn’t set any autodiscovery on our plugin so we’ll need to update our application manifests to tell them to use our new plugin. 

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: example
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  destination:
    name: ''
    namespace: example
    server: 'https://kubernetes.default.svc'
  source:
    path: user/manifests/app-name/env-name/
    repoURL: 'https://github.com/todaywasawesome/your-repo-here'
    targetRevision: HEAD
    plugin: 
      name: kustomize-build-with-helm
  project: elite-cluster
  syncPolicy:
    automated: null

Notice we added a `plugin` which references the name we have been using.

If you’re updating an existing application, make sure to do a “Hard Refresh” by navigating to the application in Argo CD, and clicking on the arrow under “Refresh” and selecting “Hard Refresh”. If you were using a previously broken plugin, Redis keeps a cache of the failure and won’t try again immediately. 

Enabling Kustomize with Helm support without CMP Plugins

Update 3/15/2024. The goal of this post is to show how to use Argo CD CMP plugins to go beyond what’s possible with simple configuration options in Argo CD but this post has become quite popular and many stumble on it when looking for info on how to configure Kustomize to --enable-helm.

This can be done globally by updating the Argo CD Configmap

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-cm
  namespace: argocd
data:
  kustomize.buildOptions: --enable-helm

Or by updating an individual Argo CD app.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
spec:
  source:
    repoURL: https://github.com/argoproj/argocd-example-apps.git
    targetRevision: HEAD
    path: kustomize-guestbook

    kustomize:
      buildOptions: --enable-helm

Read more about customing Kustomize in Argo CD from the docs.

Conclusion

Configuring Argo CD CMPs is pretty straightforward once you understand the sidecar model. If you’re still learning Argo CD, or even already a total expert, I recommend doing the Codefresh GitOps with Argo Certification, which will teach you not just the ins and outs of Argo CD but also how to use it in a GitOps-friendly way!

And of course, try out codefresh.io – it’s a lot more than just Enterprise Argo, it brings CI/CD, and GitOps together in a single platform to make software delivery exceptionally easy. Try it out free and let us know if you have any questions!

How useful was this post?

Click on a star to rate it!

Average rating 5 / 5. Vote count: 3

No votes so far! Be the first to rate this post.

One thought on “Using Argo CDs new Config Management Plugins to Build Kustomize, Helm, and More

  1. Hi

    Thank you for your blog.

    Maybe have you used private repository like oci?

    I use kustomization and helm on the argo but I have been issue oci.

    My lab is okay but when I use argo It is not okay.

    So If you were you know this issue Let me know how to fixed please.

    I can’t speak English well maybe If you feel bad about my question. I’m so sorry.

    ‘helm pull –untar –untardir /tmp/_cmp_server/027c2e5c-09bf-4553-b015-a01de99885bd/nginx/charts –repo oci://harbor-infra.huntedhappy.kro.kr/helm/ nginx –version 15.4.4’ with env=[HELM_CONFIG_HOME=/tmp/kustomize-helm-854369474/helm HELM_CACHE_HOME=/tmp/kustomize-helm-854369474/helm/.cache HELM_DATA_HOME=/tmp/kustomize-helm-854369474/helm/.data] (is ‘helm’ installed?): exit status 1

    Thanks.

Leave a Reply

Your email address will not be published. Required fields are marked *

Comment

Build your GitOps skills and credibility today with a GitOps Certification.

Get GitOps Certified

Ready to Get Started?
  • safer deployments
  • More frequent deployments
  • resilient deployments