Create your FREE Codefresh account and start making pipelines fast. Create Account

Using Argo CD and Kustomize for ConfigMap Rollouts

5 min read

Kubernetes offers a way to store configuration files and manage them via a ConfigMap. Functionally, they seem very similar to Kubernetes Secrets, where both constructs are used to store information that can be used in a Pod. This information could be usernames and passwords of a connection string to a database. While Secrets focus on key/value pairs, ConfigMaps are used to store configuration information for a specific application; for example, the application.properties configuration for a Springboot application.

While storing the configuration information in a configMap is a convenient way to distribute the application’s configuration, it does pose a problem. Changing a configuration doesn’t trigger a rollout of your application, thus leaving you to do this manually or as a part of your CI/CD process. So you can change the configuration of your application, and your application won’t read this new configuration until after it has been restarted. Ideally, the application should refresh its config itself, but since this needs development work, we’ll focus on alternatives.

Current Approaches

This introduced a challenge as more and more people started to adopt Kubernetes and changes to an application’s behavior was made by changing the configuration in the ConfigMap.

This led to many organizations and people to develop plugins to make this possible. The two most popular tools are Reloader and Configurator. They each deal with their own specific use case differently, but the idea is the same. Reloading/redeploying an application when a ConfigMap gets updated.

While these tools offer a lot, you can trigger a new rollout in another (Kubernetes native) way, if that’s all you’re looking for.

Kustomize to the Rescue

Kustomize is the native Kubernetes patching framework. This allows you to do things like reuse YAML and only store deltas between environments, merge YAML files en masse, and set common configuration settings. One of which is an option called commonAnnotations.

The commonAnnotations setting allows you to define annotations that should go in every resource that Kustomize is managing. Take the following Kustomize layout.

$ tree app
app
├── cm.yaml
├── deploy.yaml
├── ingress.yaml
├── kustomization.yaml
├── ns.yaml
└── service.yaml

 

In this layout, you’ll see something like this, in the kustomization.yaml file.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: myns

resources:
- ns.yaml
- cm.yaml
- service.yaml
- deploy.yaml
- ingress.yaml

commonAnnotations:
  sample-go/version: v1

 

As you can see, there is a section for commonAnnotations, where I’m keeping track of the changes I’m making in an annotation. This is especially useful in a GitOps workflow as someone can reject a Pull request if I somehow forgot to increment the annotation per my organization’s policy.

Why is having commonAnnotations set important? Because a change in an annotation triggers a rollout of a Deployment, Statefulset, or DaemonSet. So when making a change to the ConfigMap, you can increment the annotation value, thus triggering a rollout of your application in Kubernetes!

Example

Let’s go through an example of this. I’ll be using this repo, so you can follow along or see this example in action. I have a sample application running on a Kubernetes cluster; this application simply reads the configuration stored in the ConfigMap.

The version of this file is stored in the ConfigMap.

apiVersion: v1
kind: ConfigMap
metadata:
  name: myconfig
  namespace: myns
data:
  test.conf: |
    title = "Example Config"
    enableAwesome = "true"
    version = "v1"

 

This application is managed by Argo CD, so an update to the Git repository will rollout a new version.

Let’s make an update to the ConfigMap and see how it behaves.

$ sed -i 's/version = "v1"/version = "v1.1"/g' cm.yaml

 

Now, let’s commit this into our Git repository.

$ git add .
$ git commit -sam "updated configmap to version v1.1"
$ git push

 

Now, let’s have Argo CD sync the changes (so we don’t have to wait for the 3 min reconciliation loop to happen)

$ argocd app sync myapp

 

Let’s reload the page and see what we get.

It still says version v1, interesting. Let’s investigate: first, let’s see if the ConfigMap got updated.

$ kubectl get cm myconfig -n myns -o jsonpath='{.data.test\.conf}'
title = "Example Config"
enableAwesome = "true"
version = "v1.1"

 

Okay, that looks good. That is what we committed to Git. Let’s check the pod itself.

$ kubectl -n myns exec -it $(kubectl get pods -n myns -l app=sample-go -o name) -- cat /etc/myapp/test.conf
title = "Example Config"
enableAwesome = "true"
version = "v1"

 

The version of the ConfigMap in the pod is still in the previous state. This is because a ConfigMap update doesn’t trigger a rollout of the Deployment that manages this Pod. For this, we can use commonAnnotations in Kustomize to fix this issue. Let’s make another update to the ConfigMap.

$ sed -i 's/version = "v1.1"/version = "v1.2"/g' cm.yaml

 

Now, let’s make sure the Annotations are set so that the Deployment rollout will be triggered.

$ kustomize edit set annotation sample-go/version:v1.2

 

If you take a look at the kustomization.yaml file, you’ll see that the commonAnnotations is now set.

$ cat kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
namespace: myns

resources:
- ns.yaml
- cm.yaml
- service.yaml
- deploy.yaml
- ingress.yaml

commonAnnotations:
  sample-go/version: v1.2

 

Let’s push this now to our Git repo.

$ git add .
$ git commit -sam "updated configmap to version v1.2"
$ git push

 

NOTE: We’re pushing directly into the Git repository for this demo. In practice, you’ll make a PR and have your process verify that the commonAnnotations are set and that they are set in accordance with your policy.

Now that that’s pushed into Git, let’s have Argo CD sync our Application.

$ argocd app sync myapp

 

If you look at the Argo CD UI, you’ll see now that a new Deployment was rolled out. (You can see by the fact that there’s two ReplicaSets now.)

Taking a look at the app on the web browser shows that the Pod now is reading the newer version of the ConfigMap.

Success! We were able to automate the Deployment rollout using Kustomize (note that you’ll have to use Kustomize in order for this to work). What’s great about this solution is that it also works with Secrets and Sealed Secrets. Having the annotation change triggers a new Deployment rollout in Kubernetes, so it’s an easy solution using built-in Kubernetes tools.

Conclusion

In this blog, I went over what ConfigMaps are and how they are used in Kubernetes to store application configurations. I also went over some of the challenges that come with updating ConfigMaps and Application rollouts. With that, I went over how you can mitigate those challenges by using Kubernetes native tooling with Argo CD.

Want to learn more tips and tricks with Argo CD? Make sure you check out the free certification course on Argo CD and GitOps by Codefresh. Here, you’ll not only learn how to use Argo CD in your GitOps workflows, but you’ll also learn valuable tips and best practices from our experts. Check it out while it’s still free!

Christian Hernandez

Christian is a well rounded technologist with experience in infrastructure engineering, systems administration, enterprise architecture, tech support, advocacy, and management. Passionate about OpenSource and containerizing the world one application at a time. He is currently a maintainer of the OpenGitOps project and a member of the Argo Project Marketing SIG. He focuses on GitOps practices, DevOps, Kubernetes, and Containers.

4 responses to “Using Argo CD and Kustomize for ConfigMap Rollouts

  1. Great article! I’m curious as to why Kustomize’s ConfigMapGenerator isn’t suggested – as in my experience, this causes a deployment to rollout in a much simpler way.

    1. Christian Hernandez says:

      Hi Sean! See my reply to Aaron. In short, ConfigMapGenerator is another great way to solve this problem, I was just trying to stay as close to the GitOps principles as possible. However, ConfigMapGenerator will also cause a deployment rollout if you’re just interested in automation.

  2. Cool.

    What about Kustomize’s configMapGenerator? It will accomplish automatic rollouts without the need to manually increment annotations.

    1. Christian Hernandez says:

      Hi Aron! First, you wouldn’t be manually incrementing annotations as this would be part of your CI system. The manual steps are for the example only. Second, The configMapGenerator is another way to solve this problem. The example in the article is trying to stay as close as to the GitOps principles (opengitops.dev) as possible.

Leave a Reply

* All fields are required. Your email address will not be published.