Streamline your container workflow with Codefresh!

[Tutorial] Deploying Kubernetes to AWS using KOPs

Kubernetes Tutorial | November 13, 2017

This tutorial will show you how to quickly and easily configure and deploy Kubernetes on AWS using a tool called kops. If you’re looking for a managed solution, we suggest using Stackpoint Cloud to do a one-click deployment of Kubernetes to AWS. We also suggest looking at our Kubernetes Cloud Hosting Comparison. Otherwise, let’s get started!

Step 1: Prepare your Host Environment

The first thing that you need to do is prepare your host environment. You’ll require a few pieces of software that let you create and manage your Kubernetes cluster in AWS.

Install Kubectl

Kubectl is the command line tool that enables you to execute commands against your Kubernetes cluster. Installation is as simple as downloading the binary, marking it as executable, and adding it to your path.

Mac (Homebrew)

Mac (manual)

Linux

Install kops

Kops is an official tool for managing production-grade Kubernetes clusters on AWS. It also supports other cloud providers as alpha features. A simple way to think about it is “kubectl for clusters” — these commands enable you to configure and build your cluster. You can download the binary directly from GitHub, or use homebrew if you are on a Mac.

Mac

Linux
First, download the latest binary from kops release page, then you’ll mark it as executable and put it on your path.

Install AWS CLI tools

The AWS CLI will let you directly interact with Amazon Web Services. The CLI is distributed as a Python package.

Install AWS CLI with Homebrew

Install AWS CLI with Pip

If you’re worried about version conflicts, consider installing the AWS CLI in a virtual environment.

Configure AWS CLI tools

After you’ve installed the AWS CLI, you need to configure it to work with your AWS account. This is easily done with the following command:

Step 2: Configure Route 53 Domains

Kops requires that your cluster use DNS. Using DNS names is much easier than remembering a set of IP addresses, and it is easier to scale your cluster as well. If you are going to eventually run multiple clusters, each cluster should have its own subdomain as well. As an example:

In our example, we’ll use Route 53 Hosted Zones to configure our cluster to use dev.yourdomain.com . From there, we’ll be able to assign subdomains to our Kubernetes clusters. You can either use AWS’s Hosted Zones documentation to configure it or simply use the following command:

Now that you’ve set up your Hosted Zone, you need to configure the NS records for the parent domain to work with your subdomain. In this scenario, yourdomain.com  is the parent domain and dev  is the subdomain. You’ll need to create an NS record for dev  in yourdomain.com . This ensures that dev.yourdomain.com  correctly resolves.

If your domain is hosted in Route 53 already, create a new record NS record, and copy and paste the name servers from the subdomain. This will allow requests to reference the hosted zone we just created.

Every hosting provider exposes this functionality, but it can be a bit complicated. To validate that you’ve done it correctly, try running dig NS dev.yourdomain.com  If it responds with 4 NS records that point to your Hosted Zone, then everything is working properly.

Step 3: Create S3 buckets for storage

The next step is to configure S3 buckets to store your Kubernetes cluster configuration. Kops will use this backend storage to persist the cluster configuration. Setting up S3 buckets is very easy. You can create it online, or use the CLI. We recommend using the CLI. Since we went with dev.yourdomain.com , let’s name our S3 bucket storage.dev.yourdomain.com .

After you run this, you’ll need to export the environment variable so that kops uses the right storage.

Optionally, you can export the variable in your bash_profile  file so you don’t have to worry about configuring it every time you need to use kops.

Step 4: Build Cluster Configuration

Now that you’ve got S3 storage configured, it’s time to build your Kubernetes cluster configuration. Execute the following command:

After this completes, your cluster’s configuration will be built, and kops will output a few interesting commands to manage your cluster. Note that your cluster hasn’t actually been built, just the configuration.

Step 5: Build your Kubernetes Cluster

Now that your cluster’s configuration is built, execute the following command to instruct kops to actually build the cluster in AWS

After a few minutes, your Kubernetes cluster will be deployed in AWS. If you make any additional configuration changes, just use kops update cluster again to push your changes.

Step 6: Add Cluster to Codefresh

Now we have a cluster up. Let’s use it! First, configure  kubectl  to work with the cluster we just made.

Now all kubectl  commands will automatically point at the cluster we just created.

Adding to Codefresh is very easy, after logging in/creating a free account, click on “Kubernetes” on the left-hand side and then “Add Cluster”.

“Click Add Provider” and select “Amazon AWS”. Expand the provider dropdown to now connect your cluster. Name it whatever you like and then to get connection information using kubectl . I’ll be using pbcopy to copy each command to the clipboard, but if you don’t have that installed you can remove that part of the command.

The output of each command will be copied to your clipboard so you can paste it into Codefresh.

Host

Certificate

Token

Click “Test” to make sure the connection is working then go back to the Kubernetes page. Click “Add Service” and add the information below

Click “Deploy” and nginx will be up and running within a few seconds.

Since we exposed it publicly, you’ll see a new ELB setup in AWS for the service we just deployed.

Conclusion

While Kops isn’t as complete a solution as a fully-managed Kubernetes service like GKE, it’s still awesome. The setup time and flexibility make Kops + AWS a great way to get up and running with a production-grade cluster. The next step is to add a real application to Codefresh. Then you can build, test, and deploy the application from a single interface. Plus we can shift all the testing left, and spin up environments on-demand for every change.

Learn more about Codefresh.

About Dan Garfield

Dan is a full-stack web developer and VP of the Marketing at Codefresh. Dan is a *nix native and all around technology enthusiast.

Reader Interactions

Enjoy this article? Don't forget to share.

Comments

  1. Can you point me at documentation for doing the codefresh-specific steps via your API? If you have a terraform provider that can do it, so much the better. I’m in the process of automating our kubernetes cluster via kops and terraform/terragrunt, and while it wouldn’t be the end of the world to have to manually create new kubernetes clusters manually via your web ui, I’d much rather do it via terraform, even if I just have to kludge one together via a script that executes locally

    • Codefresh doesn’t have any tooling for creating clusters, you would just add the cluster to Codefresh once it’s been created. It’s a good feature request though. I created an issue for it.

  2. Just use Terraform and Rancher. Does all the heavy lifting for fully automated repeatable provisioning of your infrastructure, and gives you a great UX abstraction from command-line K8s.

  3. Hi Dan,

    Could you please explain, how could we create a kubernetes cluster on AWS with reserved instances using KOPS.

    • I don’t believe you’d need to do anything different when using reserved instances. Just set your master and node sizes to your reserved instances sizes. See this thread.

Reply This Comment: Fraser Goffin Cancel reply

Your email address will not be published. Required fields are marked *

Follow me on Twitter