Starting with Argo CD 2.4, creating config management plugins or CMPs via configmap has been deprecated, with support fully removed in Argo CD 2.8. While many folks have been using their own config management plugins to do things like `kustomize –enable-helm`, or specify specific version of Helm, etc – most of these seem to have not noticed the old way of doing things has been removed until just now!
The documentation has a handy guide for migrating plugins but it leaves it to the user to decide what kind of CMP they want to build. In this post, I’ll show you how to build a CMP to run Kustomize with Helm (a very popular pattern).
You can find this example, and hopefully others soon, in the repo todaywasawesome/argocd-cmp-plugin-examples. For this plugin, an example installation is provided for
- Codefresh’s Helm Chart
- Community Helm Chart
- Plain Kubernetes Manifests
- Kustomize with Argo CD Autopilot.
How CMPs work in Argo CD
The CMP system works by adding an arbitrary container sidecar to `argocd-repo-server` and passing in configuration and mounting the CMP server service. Repos are checked out using repo-server and then mounted into the CMP sidecar to allow rendering. Any CMP you create will ultimately have 1 job, render manifests that Argo CD can apply. This is very easy to set up!
Step 1 – Create a Configuration for our CMP
First, we need to create a configuration for our CMP. Our plugin will be called `kustomize-build-with-helm`. Here’s what it looks like:
apiVersion: v1
kind: ConfigMap
metadata:
name: kustomize-build-with-helm
data:
plugin.yaml: |
apiVersion: argoproj.io/v1alpha1
kind: ConfigManagementPlugin
metadata:
name: kustomize-build-with-helm
spec:
generate:
command: [ "sh", "-c" ]
args: [ "kustomize build --enable-helm" ]
That file looks funny, doesn’t it?
CMPs use a file that looks just like a Kubernetes CRD called `ConfigManagementPlugin` but actually, it’s just a yaml file that we need mounted in our sidecar. To do this we’re going to create a `ConfigMap` and mount it into our CMP sidecar. You can also bake this file into your sidecar image if you would like.
CMPs support a ton of additional options like:
- Automatically discovering when the plugin should be run by using a glob expression to find files
- Running init commands every time the CMP runs
- Creating parameters that can be exposed and configured in the Argo CD UI
- And lots more, check out the example file for all parameters.
Our plugin is very simple by contrast, it’s only going to build manifests using `kustomize build –enable-helm` and that’s it.
Step 2 – Create a CMP Sidecar
The sidecar has to have the same name you specified in the config file we just created. It also will need a Docker image to run that contains our tools. Installing tools during the `init` step is not recommended because it runs every time a CMP runs. For the sake of simplicity, we’re going to use `alpine/k8s` a ~220MB image that containers kubectl, helm, kustomize and a few other tools. If course you can build your own slim image but at the time of writing, no one has offered their own images for Argo CD CMP plugins.
Here’s what our sidecar will look like:
containers:
- name: kustomize-build-with-helm
command: [/var/run/argocd/argocd-cmp-server] # Entrypoint should be Argo CD lightweight CMP server i.e. argocd-cmp-server
# image: busybox # This can be off-the-shelf or custom-built image
image: alpine/k8s:1.26.8
securityContext:
runAsNonRoot: true
runAsUser: 999
volumeMounts:
- mountPath: /var/run/argocd
name: var-files
- mountPath: /home/argocd/cmp-server/plugins
name: plugins
# Remove this volumeMount if you've chosen to bake the config file into the sidecar image.
- mountPath: /home/argocd/cmp-server/config/plugin.yaml
subPath: plugin.yaml
name: kustomize-build-with-helm
# Starting with v2.4, do NOT mount the same tmp volume as the repo-server container. The filesystem separation helps
# mitigate path traversal attacks.
- mountPath: /tmp
name: cmp-tmp
volumes:
- configMap:
name: kustomize-build-with-helm
name: kustomize-build-with-helm
- emptyDir: {}
name: cmp-tmp
This container will need to be added to our `argocd-repo-server` deployment and I’ll show how to do that using Kustomize in just a moment.
Let’s take a moment to explore this file.
We specify our image `alpine/k8s:1.26.8` – of course you’ll want this to match your Kubernetes version. If you’re supporting multiple versions of Kubernetes, then you can create an additional CMP plugin for each version!
Then we mount several paths, first we mount two volumes that will run our Argo CD CMP service so repo-server can communicate with the sidecar. Next, we mount the `ConfigMap` we created earlier. Finally, we mount an empty directory to serve as a temp folder. It’s important that we not mount the same temp folder `argocd-repo-server` uses as that would make path traversal attacks easier. (Read more about running Argo CD securely).
Step 3 – Add our CMP to Argo CD (bonus, using Kustomize!)
To install this we need to modify our Argo CD instance. We can either edit the Kubernetes manifests directly to add the sidecar to the `argocd-repo-server` deployment, edit our Helm values, or we can add it in using Kustomize (see example repo).
For this blog, I’m using Argo CD Autopilot which uses Kustomize to install Argo CD.
First, I’ll create a file for my `configmap` and name it `kustomize-with-helm.configmap.yaml`.
apiVersion: v1
kind: ConfigMap
metadata:
name: kustomize-build-with-helm
data:
plugin.yaml: |
apiVersion: argoproj.io/v1alpha1
kind: ConfigManagementPlugin
metadata:
name: kustomize-build-with-helm
spec:
generate:
command: [ "sh", "-c" ]
args: [ "kustomize build --enable-helm" ]
Then I’ll create a file for my sidecar and name it `kustomize-with-helm-sidecar.yaml`
apiVersion: apps/v1
kind: Deployment
metadata:
name: argocd-repo-server
spec:
template:
spec:
containers:
- name: kustomize-build-with-helm
command: [/var/run/argocd/argocd-cmp-server] # Entrypoint should be Argo CD lightweight CMP server i.e. argocd-cmp-server
# image: busybox # This can be off-the-shelf or custom-built image
image: alpine/k8s:1.26.8
securityContext:
runAsNonRoot: true
runAsUser: 999
volumeMounts:
- mountPath: /var/run/argocd
name: var-files
- mountPath: /home/argocd/cmp-server/plugins
name: plugins
# Remove this volumeMount if you've chosen to bake the config file into the sidecar image.
- mountPath: /home/argocd/cmp-server/config/plugin.yaml
subPath: plugin.yaml
name: kustomize-build-with-helm
# Starting with v2.4, do NOT mount the same tmp volume as the repo-server container. The filesystem separation helps
# mitigate path traversal attacks.
- mountPath: /tmp
name: cmp-tmp
volumes:
- configMap:
name: kustomize-build-with-helm
name: kustomize-build-with-helm
- emptyDir: {}
name: cmp-tmp
Finally, I’ll update my `kustomization.yaml` to create the new configmap and patch in my sidecar. I’ve removed some config from this file for the sake of brevity. If you’re using Argo CD Autopilot it will also have a configmap generator to create the git credentials.
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: argocd resources: - github.com/argoproj-labs/argocd-autopilot/manifests/base?ref=v0.4.15 - kustomize-with-helm.yaml # Adds our configmap patches: - path: kustomize-with-helm.sidecar.yaml # Adds the sidecar to argocd-repo-server
Now we can commit that, or run a `kubectl apply -k .` on the Kustomization we just created.
Step 4 – Configure Applications to use our Plugin
Finally, we didn’t set any autodiscovery on our plugin so we’ll need to update our application manifests to tell them to use our new plugin.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: example
finalizers:
- resources-finalizer.argocd.argoproj.io
spec:
destination:
name: ''
namespace: example
server: 'https://kubernetes.default.svc'
source:
path: user/manifests/app-name/env-name/
repoURL: 'https://github.com/todaywasawesome/your-repo-here'
targetRevision: HEAD
plugin:
name: kustomize-build-with-helm
project: elite-cluster
syncPolicy:
automated: null
Notice we added a `plugin` which references the name we have been using.
If you’re updating an existing application, make sure to do a “Hard Refresh” by navigating to the application in Argo CD, and clicking on the arrow under “Refresh” and selecting “Hard Refresh”. If you were using a previously broken plugin, Redis keeps a cache of the failure and won’t try again immediately.
Enabling Kustomize with Helm support without CMP Plugins
Update 3/15/2024. The goal of this post is to show how to use Argo CD CMP plugins to go beyond what’s possible with simple configuration options in Argo CD but this post has become quite popular and many stumble on it when looking for info on how to configure Kustomize to --enable-helm.
This can be done globally by updating the Argo CD Configmap
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
namespace: argocd
data:
kustomize.buildOptions: --enable-helmOr by updating an individual Argo CD app.
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: guestbook
spec:
source:
repoURL: https://github.com/argoproj/argocd-example-apps.git
targetRevision: HEAD
path: kustomize-guestbook
kustomize:
buildOptions: --enable-helmRead more about customing Kustomize in Argo CD from the docs.
Conclusion
Configuring Argo CD CMPs is pretty straightforward once you understand the sidecar model. If you’re still learning Argo CD, or even already a total expert, I recommend doing the Codefresh GitOps with Argo Certification, which will teach you not just the ins and outs of Argo CD but also how to use it in a GitOps-friendly way!
And of course, try out codefresh.io – it’s a lot more than just Enterprise Argo, it brings CI/CD, and GitOps together in a single platform to make software delivery exceptionally easy. Try it out free and let us know if you have any questions!