Kubernetes offers a way to store configuration files and manage them via a ConfigMap. Functionally, they seem very similar to Kubernetes Secrets, where both constructs are used to store information that can be used in a Pod. This information could be usernames and passwords of a connection string to a database. While Secrets focus on key/value pairs, ConfigMaps are used to store configuration information for a specific application; for example, the application.properties configuration for a Springboot application.
While storing the configuration information in a configMap is a convenient way to distribute the application’s configuration, it does pose a problem. Changing a configuration doesn’t trigger a rollout of your application, thus leaving you to do this manually or as a part of your CI/CD process. So you can change the configuration of your application, and your application won’t read this new configuration until after it has been restarted. Ideally, the application should refresh its config itself, but since this needs development work, we’ll focus on alternatives.
Current Approaches
This introduced a challenge as more and more people started to adopt Kubernetes and changes to an application’s behavior was made by changing the configuration in the ConfigMap.
This led to many organizations and people to develop plugins to make this possible. The two most popular tools are Reloader and Configurator. They each deal with their own specific use case differently, but the idea is the same. Reloading/redeploying an application when a ConfigMap gets updated.
While these tools offer a lot, you can trigger a new rollout in another (Kubernetes native) way, if that’s all you’re looking for.
Kustomize to the Rescue
Kustomize is the native Kubernetes patching framework. This allows you to do things like reuse YAML and only store deltas between environments, merge YAML files en masse, and set common configuration settings. One of which is an option called commonAnnotations.
The commonAnnotations setting allows you to define annotations that should go in every resource that Kustomize is managing. Take the following Kustomize layout.
$ tree app app ├── cm.yaml ├── deploy.yaml ├── ingress.yaml ├── kustomization.yaml ├── ns.yaml └── service.yaml
In this layout, you’ll see something like this, in the kustomization.yaml file.
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: myns resources: - ns.yaml - cm.yaml - service.yaml - deploy.yaml - ingress.yaml commonAnnotations: sample-go/version: v1
As you can see, there is a section for commonAnnotations, where I’m keeping track of the changes I’m making in an annotation. This is especially useful in a GitOps workflow as someone can reject a Pull request if I somehow forgot to increment the annotation per my organization’s policy.
Why is having commonAnnotations set important? Because a change in an annotation triggers a rollout of a Deployment, Statefulset, or DaemonSet. So when making a change to the ConfigMap, you can increment the annotation value, thus triggering a rollout of your application in Kubernetes!
Example
Let’s go through an example of this. I’ll be using this repo, so you can follow along or see this example in action. I have a sample application running on a Kubernetes cluster; this application simply reads the configuration stored in the ConfigMap.
The version of this file is stored in the ConfigMap.
apiVersion: v1
kind: ConfigMap
metadata:
name: myconfig
namespace: myns
data:
test.conf: |
title = "Example Config"
enableAwesome = "true"
version = "v1"
This application is managed by Argo CD, so an update to the Git repository will rollout a new version.
Let’s make an update to the ConfigMap and see how it behaves.
$ sed -i 's/version = "v1"/version = "v1.1"/g' cm.yaml
Now, let’s commit this into our Git repository.
$ git add . $ git commit -sam "updated configmap to version v1.1" $ git push
Now, let’s have Argo CD sync the changes (so we don’t have to wait for the 3 min reconciliation loop to happen)
$ argocd app sync myapp
Let’s reload the page and see what we get.
It still says version v1, interesting. Let’s investigate: first, let’s see if the ConfigMap got updated.
$ kubectl get cm myconfig -n myns -o jsonpath='{.data.test.conf}'
title = "Example Config"
enableAwesome = "true"
version = "v1.1"
Okay, that looks good. That is what we committed to Git. Let’s check the pod itself.
$ kubectl -n myns exec -it $(kubectl get pods -n myns -l app=sample-go -o name) -- cat /etc/myapp/test.conf title = "Example Config" enableAwesome = "true" version = "v1"
The version of the ConfigMap in the pod is still in the previous state. This is because a ConfigMap update doesn’t trigger a rollout of the Deployment that manages this Pod. For this, we can use commonAnnotations in Kustomize to fix this issue. Let’s make another update to the ConfigMap.
$ sed -i 's/version = "v1.1"/version = "v1.2"/g' cm.yaml
Now, let’s make sure the Annotations are set so that the Deployment rollout will be triggered.
$ kustomize edit set annotation sample-go/version:v1.2
If you take a look at the kustomization.yaml file, you’ll see that the commonAnnotations is now set.
$ cat kustomization.yaml apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization namespace: myns resources: - ns.yaml - cm.yaml - service.yaml - deploy.yaml - ingress.yaml commonAnnotations: sample-go/version: v1.2
Let’s push this now to our Git repo.
$ git add . $ git commit -sam "updated configmap to version v1.2" $ git push
NOTE: We’re pushing directly into the Git repository for this demo. In practice, you’ll make a PR and have your process verify that the commonAnnotations are set and that they are set in accordance with your policy.
Now that that’s pushed into Git, let’s have Argo CD sync our Application.
$ argocd app sync myapp
If you look at the Argo CD UI, you’ll see now that a new Deployment was rolled out. (You can see by the fact that there’s two ReplicaSets now.)
Taking a look at the app on the web browser shows that the Pod now is reading the newer version of the ConfigMap.
Success! We were able to automate the Deployment rollout using Kustomize (note that you’ll have to use Kustomize in order for this to work). What’s great about this solution is that it also works with Secrets and Sealed Secrets. Having the annotation change triggers a new Deployment rollout in Kubernetes, so it’s an easy solution using built-in Kubernetes tools.
Conclusion
In this blog, I went over what ConfigMaps are and how they are used in Kubernetes to store application configurations. I also went over some of the challenges that come with updating ConfigMaps and Application rollouts. With that, I went over how you can mitigate those challenges by using Kubernetes native tooling with Argo CD.
Want to learn more tips and tricks with Argo CD? Make sure you check out the free certification course on Argo CD and GitOps by Codefresh. Here, you’ll not only learn how to use Argo CD in your GitOps workflows, but you’ll also learn valuable tips and best practices from our experts. Check it out while it’s still free!