Dynamically Creating K8s Namespaces for Every Branch/Pull Request

Dynamically Creating K8s Namespaces for Every Branch/Pull Request

4 min read

Shift Left with Kubernetes and Helm

One of the important principals of shifting left is having the ability to create an environment for each branch you’re working on. Shifting left is valuable because it puts application engineers in a position where they can totally validate changes before they even issue a pull request. It saves time for people doing code reviews and approving pull requests because they have a lot more information about how valid and well tested a change is.

Plus it’s nice to be able to share changes with your teams before.

Making it happen in Kubernetes

Today I’ll show you how to

  • Build a new Docker image
  • Dynamically create a namespace
  • Copy secrets to the new namespace
  • Push images (in this case using a Helm Chart)
  • Put all of that into a Codefresh pipeline

I’ll start by showing the pipeline and then break it down.

version: '1.0'
steps:
  BuildingDockerImage:
    title: Building Docker Image
    type: build
    image_name: containers101/demochat
    working_directory: ./
    dockerfile: Dockerfile
    tag: '${{CF_BRANCH_TAG_NORMALIZED}}'
  createNamespace:
    title: Create new namespace
    image: codefresh/kube-helm:master
    fail_fast: false
    commands:
     - kubectl config use-context ${{KUBE_CONTEXT}}
     - kubectl create namespace ${{CF_BRANCH_TAG_NORMALIZED}}
     - kubectl get secret codefresh-generated-r.cfcr.io-cfcr-default -o json | jq '.metadata.namespace = "${{CF_BRANCH_TAG_NORMALIZED}}"' | kubectl create --namespace ${{CF_BRANCH_TAG_NORMALIZED}} -f -
  release_to_env:
    title: Deploying Helm Chart
    image: 'codefresh/plugin-helm:2.7.2'

Building the Docker Image

Building the Docker image is pretty straight forward. We pass a name, the working directory, the location of the Dockerfile relative to repository and give it any tags we’d like it to have. Codefresh will take care of the rest.

    title: Building Docker Image
    type: build
    image_name: containers101/demochat
    working_directory: ./
    dockerfile: Dockerfile
    tag: '${{CF_BRANCH_TAG_NORMALIZED}}'

Take notice of the variable ${{CF_BRANCH_TAG_NORMALIZED}}. Git branch names can have all kinds of characters that Docker and Kubernetes don’t like. The “normalized” version will replace all those funky characters with something more palatable.

If this pipeline is running on the branch “new feature” it will create the image “demochat:new-feature”.

Creating the Namespace

To create the namespace we’re going to use a custom step. It will take an image called “kube-helm” which has Kubectl, Helm, jq and a few other tools installed.

createNamespace:
    title: Create new namespace
    image: codefresh/kube-helm:master
    fail_fast: false
    commands:
     - kubectl config use-context ${{KUBE_CONTEXT}}
     - kubectl create namespace ${{CF_BRANCH_TAG_NORMALIZED}}

Here we’ve set an environmental variable on the pipeline called ${{KUBE_CONTEXT}} . This is the name of the cluster that I’ve already added to Codefresh. If you haven’t added a cluster to Codefresh, it’s really easy.

The create namespace line will again use our ${{CF_BRANCH_TAG_NORMALIZED}} variable.

We’ll use the fail_fast option since this step will throw an error if the namespace was already created. The fail_fast will let the rest of the pipeline run even if that happens (as it will every time after the first time we create a new branch).

So far this is pretty easy! The real trick will be adding our secrets.

Copying Secrets between Namespaces

kubectl get secret codefresh-generated-r.cfcr.io-cfcr-default -o json | jq '.metadata.namespace = "${{CF_BRANCH_TAG_NORMALIZED}}"' | kubectl create --namespace ${{CF_BRANCH_TAG_NORMALIZED}} -f -

This is actually several commands piped together. The first command will get a secret from our default namespace that I’ve previously saved.

If you wanted to specify a specific namespace to pull a secret from then you would just throw the –namespace=whatever option on. You will need to know the name of the secret you want to copy from. Here, I used Codefresh to create the secret earlier and I’m going to reuse it. codefresh-generated-r.cfcr.io-cfcr-default

Secrets contain a reference for which namespace they should work in. When we copy the secret we need to modify it. We’ll pipe the output as json to a tool called jq which can parse json and use it to modify our secret for a new namespace. jq ‘.metadata.namespace = “${{CF_BRANCH_TAG_NORMALIZED}}”‘ .

Finally, we’ll apply the secret to our namespace by piping the whole thing to this line. kubectl create –namespace ${{CF_BRANCH_TAG_NORMALIZED}} -f – . This will take the secret we just copied, then modified and apply it to our new namespace.

Wondering what a secret is?

Pull secrets let Kubernetes access private Docker registries, like the one built-in to Codefresh. The image that we built at the top of this post was a private Docker image so we need to make sure Kubernetes can pull it when we goto deploy.

Deploy Built Image using a Helm Chart

Now that we’ve built a Docker image, and dynamically created a namespace we can deploy it along with it’s Chart. We’ll use the Codefresh Helm plugin with the following YAML.

  release_to_env:
    title: Deploying Helm Chart
    image: 'codefresh/plugin-helm:2.7.2'

You can get the full documentation here. Basically this step, plus a few environmental variables will take everything we did and deploy it. Here’s a screenshot of my environmental variables so you can see how I used Codefresh variables to pull in the new image, configure the Helm deploy step etc.

And here’s what the whole thing looks like when I run it:

As a bonus, here’s a video of me using this in a feature friday video!

New to Codefresh? Create Your Free Account Today!

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

Leave a Reply

Your email address will not be published. Required fields are marked *

Comment

Build your GitOps skills and credibility today with a GitOps Certification.

Get GitOps Certified

Ready to Get Started?
  • safer deployments
  • More frequent deployments
  • resilient deployments