Streamline your container workflow with Codefresh!

Kubernetes in the Cloud: AWS vs. GCP vs. Azure

Kubernetes Tutorial | July 21, 2017

*** Limited time offer: Create a free Codefresh Account and get $500 GCP Credit ***

Kubernetes is a powerful container orchestrator that allows automating, scaling, and managing container deployments. We have already covered Kubernetes basics in the past, check out that post if you want to learn more.

In this comparison we’ll look at how easy Kubernetes is to setup and manage as well as the costs and features in the big cloud. Plus we’ll look at how easy they are to integrate into your Kubernetes deployment flow with Codefresh.

Amazon Web Services (AWS), Google Cloud Platform (GCP), and Microsoft Azure all have services that are capable of setting up Highly Available installations of Kubernetes. Let’s take a look at the pros and cons of each cloud.

Amazon Web Services (AWS)

AWS is the most mature public cloud, and many businesses already have a well established presence on AWS. AWS has their own proprietary container orchestrator, Elastic Container Service (ECS), but it is different than Kubernetes.

The project Kubernetes Operations (kops) has become the de-facto standard for creating, upgrading, and managing Kubernetes clusters on AWS. Kops is a well maintained open source project with an active community.

Pros: (kops)

  • Deploy HA Kubernetes master nodes
  • Upgrade Kubernetes master nodes
  • Upgrade, add, or remove worker nodes
  • kops is highly configurable
  • Supported by Codefresh

Cons:

  • No support for Kubernetes in AWS management console; must use kops or another third-party service
  • kops configurations need to be managed and backed up

Google Cloud Platform (GCP)

Google Container Engine (GKE) is Google Cloud Platform’s managed Kubernetes service. Google is the #1 contributor to the Kubernetes open source project and GKE has been around since the early (pre 1.0) days of Kubernetes.

Google manages the Kubernetes master nodes for you in GKE, meaning that you don’t have access to them but you also aren’t charged for their compute resources. If your GKE cluster grows above 5 nodes, it costs around $100 per month (in addition to the worker node compute resources), which is on-par with what it would cost to run your own master nodes.

Another feature that sets GCP apart is that they provide a global spanning load balancer built-in which is autoconfigured when services are created. This load balancer is essentially the same that handles Youtube.com and Google.com. On Amazon and Azure the load balancer is just another container instance which you need to scale.

Pros:

  • Abstracts away and manages Kubernetes master nodes
  • Nodes use container optimized image
  • Upgrade Kubernetes Master through GCP management console
  • Upgrade, add, or remove worker nodes through GCP management console
  • Global spanning load balancer built-in
  • Easy deployment and automation
  • Optional support for Kops
  • Streamlined integration with Codefresh

Cons:

  • Unable to login to Kubernetes master nodes and change advanced settings*

*This con is a little unfair Kops supports Google Cloud which means you can always go beyond the managed service to build your own master nodes.

Microsoft Azure

Azure Container Service (ACS) allows for quick deployment of Kubernetes to Microsoft Azure. ACS is the newest tool of the three clouds to support Kubernetes.

ACS acts as more of a deployment template since it does not include features to upgrade a cluster after it has been deployed. In order to upgrade clusters using ACS, you would need to use ACS to create a new cluster, migrate containers to it, and remove the old cluster.

One bonus feature of Azure is that creating templates out of your infrastructure is very easy. Once you create and configure all your services you can essentially take a snapshot and deploy it again. Those familiar with Amazon’s Cloud Formation Templates will find Azure’s system much easier to use.

Pros:

  • Deploy HA Kubernetes master nodes
  • Add or remove worker nodes through Azure management console
  • Supported by Codefresh
  • Save infrastructure definition

Cons:

  • Upgrading Kubernetes nodes is a manual process

Which is the Best?

Features

Kubernetes Cloud Hosting Feature Comparison
CloudDeploy HA Master NodesUpgrade Master NodesDirect ManagementAbstract ManagementOptimized ImagesIntegration w/CodefreshStreamlined Integration
Google Cloud Platform✔*
Amazon Web Services✔*✔*✔*
Microsoft AzureAdd, Remove

Google Container Engine (GKE) is the most feature-rich public cloud service available today for running Kubernetes. It allows you to create, upgrade, and scale HA Kubernetes clusters using just the tools provided within GKE.

Deploying Kubernetes to AWS using kops will provide some of the maintainability and scalability of GKE without the intuitive ease of use. The only downside here is that kops is an open source tool and not a part of AWS, so the integration is not as nice as GKE.

Azure Container Service (ACS) is the youngest of the three and has the furthest to go. We’d like to see the ability to upgrade masters and nodes before recommending it for production, but it’s great to tryout for development purposes today.

Cost

One thing we haven’t exactly mentioned is cost. How much does it cost to run your production Kubernetes cluster on GKE, AWS and Azure? For this, we’ll make a few assumptions and then show how the costs of each cloud platform compare:

  1. 5 master nodes
  2. 15 worker nodes
  3. Each node requires 4 vCPU and 16 GB of RAM

Amazon Web Services

To run a production cluster on AWS, we want to stick with m4.xlarge nodes. These nodes cost $0.20 per hour. With a 20 node cluster, your deployment will have 14,440 compute hours which will cost $2,880 per month.

Google Cloud Platform

GCP gives several advantages that affect pricing. For example, they handle the manager nodes for you, and they don’t count towards your node billing. In addition, if you run your cluster the entire month, you get a flat 30% discount. For this scenario, we need 15 n1-standard-4 machines which cost $0.19 per hour. With a 15 node cluster, your deployment will have 10,800 compute hours which will cost $2,052 per month before discounts. After the 30% discount, the cluster will only cost $1,539. There is an additional fee to manage the master nodes. This is $109.50, so your entire cluster will cost $1648.50.

Microsoft Azure

For production workloads, Microsoft Azure recommends using their D2-64-v3 systems. Specifically for our deployment, we will need 20 D4-V3 machines, which cost $0.20 per hour. With a 20 node cluster, your deployment will have 14,440 compute hours which will cost $2,880 per month.

Cost Comparison

Kubernetes Hosting Cost Comparison (Monthly)
Amazon Web ServicesGoogle Cloud PlatformMicrosoft Azure
20 x m4.xlarge nodes
(80 CPUs, 320GB RAM)
20 x n1-standard-4 nodes
(80 CPUs, 320GB RAM)
20x D2-64-v3 nodes
(80 CPUs, 320GB RAM)
$2,880/mo$1,648.50/mo$2,880/mo

Kubernetes on a budget

Of course, you can run Kubernetes with a lot less hardware but it’s hard to configure a fair comparison for a lower price because Google Cloud just provides so much more RAM by default. The smallest option on Google Cloud has 3.75GBs of RAM and costs around $11/mo. Below 5 nodes Google Cloud waives fees related to cluster management which on other platforms would be part of your normal compute pricing.

Other Clouds

There are far more cloud options than AWS, GCP, and Azure. Kubernetes should be able to work on any cloud that provides multiple virtual machines and networking between them.

Integration with Codefresh

Codefresh supports automated deployment with any Kubernetes cluster. But there are some special benefits to working with Google Container Engine because there is a streamlined integration.

  • Login with Google to connect clusters
  • Better support for cluster monitoring
  • Easier cluster management and manual deployment

Read more about how Codefresh works with Google Container Engine.

Conclusion

If you’re working with Kubernetes Google Container Engine wins in almost every category, it has more features, cheaper pricing, slicker integrations, and lots of free addons. It’s also the easiest of the three to use.

Even though Microsoft, and Amazon have joined Google in the Cloud Native Computing Foundation as backers of Kubernetes, Kubernetes development is still driven by Google. They invented it so it’s no surprise they offer the best option for hosting Kubernetes today.

*** Limited time offer: If you decide to try Google Cloud Platform, Create a free Codefresh Account and get $500 GCP Credit ***

About Caleb Lloyd

Reader Interactions

Enjoy this article? Don't forget to share.

Follow me on Twitter