What is Infrastructure-As-Apps, Infra-as-Apps, or IaA
Infrastructure-as-apps builds on infrastructure-as-code to a logical endpoint by bringing in principles of GitOps management. The term is something I coined
in 2021 to describe an existing movement to bring infrastructure into the same lifecycle control as applications under GitOps. Examples of Infra-as-apps tools include Argo CD, Crossplane, Cluster API, Cello, or even SchemaHero for databases and the list is always growing.
Some of the benefits of infra-as-apps include
- Dramatically easier infrastructure management
- Better security
- Easier onboarding and resource provisioning for devs
- Faster mean time to recover (MTTR)
- Ironclad auditability
- Better cost management
- Cross-cloud and easier migration
Read on to understand why.
What an app means in infra-as-apps
Argo CD has the concept of an application that includes a desired state configuration along with the destination for that configuration and policies for syncing and managing these resources. When an application is deployed Argo CD constantly monitors the actual state of the application and compares it to the desired state. When divergence occurs, either because of drift (a change to the actual state) or because of a new desired state, Argo CD will attempt to reconcile. This is the GitOps style of deployment and follows the Open GitOps standards. If you need a primer on GitOps, we wrote a great one, read our GitOps primer here.
For most engineering teams, apps have meant the software they run on top of infrastructure. A typical flow would be someone creating the infrastructure, either manually or using tools like Terraform or Cloud Formation Templates, and then another team comes along with their apps to be deployed.
Getting the infra-as-apps
Treating infrastructure like apps in the GitOps sense has long been incredibly difficult. Creating a simple declarative configuration has become very popular with tools like Terraform, Cloud Formation templates, Pulumi, and others. But these declarative formats are generally applied once or applied on change to the desired state via CI/CD pipelines. The problem with this approach is it is missing the back half of the GitOps equation. Namely, CI/CD pipelines don’t monitor the state for drift. A CI/CD pipeline may push a new Terraform plan, but if someone modifies the actual state afterward, a CI/CD system is unaware. This is where you need a GitOps operator like Argo CD to monitor the actual state and trigger reconciliation. This “no drift allowed” approach is already helping teams manage their applications much more efficiently, and helping them recover from failures dramatically faster. After all, a rollback is just a git revert.
Now, it’s possible to get the same benefits teams have enjoyed for managing their apps with their infrastructure.
At the top of this blog post, there are several tools mentioned that enable infra-as-apps. However, we’re going to focus this post on Crossplane. Crossplane is an open-source, CNCF project that allows users to provision and manage any kind of cloud resource via the Kubernetes API. When combined with a GitOps operator like Argo CD this means you can deploy an app that represents EC2 instances, S3 buckets, Databases, Kubernetes clusters, or Load Balancers on AWS, GCP, Azure, and many other clouds.
Infrastructure-as-apps in the real world, at Cern.
At GitOpsCon US 2021, Ricardo Rocha showed just how powerful infra-as-apps can be. Cern is famously known as the organization that runs the Large Hadron Collider, the most powerful particle Accelerator. To help them process the data done in experiments and provide services to their researchers, they use over 600 clusters, 3000 nodes, 13,000 cores, 30 TBs of Ram, and 160 TBs of raw storage.
You can watch the full talk here.
In their implementation, Cern provides Helm charts that include Crossplane configuration for their clusters along with the common resources to be deployed on that cluster. Users can provision a new cluster with a simple PR and then extend the resources on the cluster with additional apps or by adding them as dependencies for the umbrella Helm chart.
One of their motivators was the ability to quickly spin up large amounts of cpus and gpus to run analysis on experiments quickly. For example, they’ll analyze 70TB of data in 5 minutes leveraging GPUs across Kubernetes. While most think of cloud resources as roughly infinite, for certain burst periods Cern will consume 10s of thousands of CPUs so they need the ability to go across regions and multi-cloud to have the bandwidth at any given movement to handle their resource spikes. Infra-as-apps makes that possible. They can describe the resources they need and the cloud provider is simply a value to pass.
But infra-as-apps isn’t limited to deploying Kubernetes resources. Infra-as-Apps means deploying any kind of resource in a GitOps way.
Infra-as-apps is a game-changer
Managing infrastructure has long been treated differently from the applications they run. There is a common division of labor between infrastructure and applications. Updates to applications are more frequent and often get more attention from CI/CD. The underlying infrastructure is often left in a set once and forget model. Automating infrastructure in a fully-GitOps compliant way is difficult so it is left behind and treated differently than the application layer.
Now all infrastructure doesn’t just get treated like code, it is actually monitored like apps via GitOps operators. This has implications across the entire development process.
Security – Permissions aren’t needed from hardly anyone
Rather than all users who need to provision resources being given keys defacto with cloud providers, those keys can be managed centrally as part of the GitOps operator. Users instead just open pull requests for their infra changes. Now users don’t really need keys to every cloud. Instead, they only need to worry about Git access.
This also means every change goes through git and can go through security, resource, and performance reviews as part of that process. When seen in action, it’s so obvious you’ll wonder how you managed infrastructure at scale without it.
Faster Repair, Upgrade, Revert
One observation from Ricardo’s talk was that it was generally easier to simply provision a new resource and shut down the old one than to bother with upgrading. When implemented fully, Infra-As-Apps means new versions of infrastructure or changes can either be updated in place via Crossplane and Argo CD, or new resources can be spun up just as quickly and leave the entire stack in a blue/green deployment state with only the load balancer requiring an update. Of course that update should happen via GitOps as well. While this is possible without infra-as-apps, as a pattern its much easier when using infra-as-apps tooling.
CI/CD to Improve Pull Requests
Instead of relying on a CI/CD process to deploy an application, monitor, and potentially roll back, CI/CD can instead focus on improving pull requests with automation via Argo CD to automatically handle rollbacks, etc.
Because the entire stack, including infrastructure, is included in a PR, better testing and policy implementation for security and resource management are much simpler.
Provisioning Custom Resources or Going Multi-Cloud is Significantly Easier
Crossplane not only makes it possible to provision resources on any cloud, but it also allows for writing custom resources that can represent resources for any cloud, abstracting away differences and making provisioning easier. For example, Viktor Farcic showed an example where he created a resource for provisioning nodes with a “small, medium, or large” option. Users asking for a small node don’t need to look up the most cost-effective node type for the size they’re looking for. Instead, the custom provider translates the desired state of “small” into the node-type for whichever cloud node is requested.
Conclusion
Infrastructure-as-apps is really a term meant to distinguish for the confusion that exists around GitOps and Infra-as-Code. It goes beyond infra-as-code to fully implement GitOps at every level. GitOps requires an operator to constantly monitor the actual state and compare it to the desired state. Most infrastructure-as-code implementations stop at a classic CI/CD approach that merely pushes changes and is oblivious to how successfully plans were applied, or if someone has made changes to production manually.
Infra-As-Apps on the other hand would view manual changes to infrastructure as a divergence from the desired state as set in git. Attackers would find changes to infra much more difficult if they’re always reverted automatically. Likewise, developers will find provisioning resources much easier, and operations teams will find changes much more predictable and the clarity of usage much easier to understand when implementing Infrastructure-as-apps.
Suggested tools:
In a future post we’ll explore how to implement Infra-as-apps in Argo CD from Codefresh Community edition (for free). In the meantime checkout this intro to Crossplane and Argo CD. Codefresh is doing for Argo what Github did for Git by providing an easy-to-use enterprise-ready implementation of Argo that is ready to scale.