Article Content
What is GitOps?
GitOps is an operational framework based on DevOps practices, like continuous integration/continuous delivery (CI/CD) and version control, which automates infrastructure and manages software deployment. It enables developers to store the desired state of their infrastructure and use it to automate operational processes. GitOps is a set of best practices applied from the beginning of the development workflow, all the way to deployment.
GitOps is a developer-centric approach, relying on tools developers are already familiar with. Developers already use Git for the source code of the application – GitOps extends this practice to application configuration, infrastructure, and operational procedures. GitOps stores every aspect of a project’s infrastructure, including infrastructure as code files, configuration files, and application code files, in Git repositories. All changes to applications and infrastructure are automatically synchronized with the live environment.
GitOps can be used to manage deployments for any infrastructure. It is especially useful for software developers and platform engineers who work with Kubernetes and want to move toward continuous operating models. GitOps makes it easy to implement continuous deployment for cloud native applications. It does this by ensuring that cloud infrastructure is immediately reproducible based on the state of a Git repository.
It is important to note that GitOps is not the next thing after DevOps or even DevOps 2.0.
GitOps is a set of deployment practices while DevOps is a paradigm or better yet, a mindset. Their shared principles make it easier for teams to adopt the GitOps workflow for existing DevOps techniques.
The 4 Commandments of GitOps
Here are the four things GitOps practitioners swear by. You absolutely need them to do GitOps right.
1. Git for Everything
GitOps extends the practice of using Git for the source code of an application to the application’s configuration, infrastructure, and operational procedures. It stores every aspect of a project’s infrastructure, including infrastructure as code files, configuration files, and application code files, in Git repositories, making it easy to manage software deployment and infrastructure provisioning.
By leveraging a declarative format, you can describe how your infrastructure works as well as the applications that are running on top of it. Doing this enables traceability of changes made to any environment and enables rollback, recoverability, and self-healing attributes using any source control system. Move away from imperative adhoc scripts to declarative configuration in all levels (app and infra).
2. V for Versioning
Declarative descriptions stored within the repository supports immutability, versioning, and versioning history. For example, using Git for the declarations mentioned above, allows you a single place from which everything for your application is derived and driven. This allows you to easily pinpoint any changes made at any time. Don’t try to understand what is different between two environments by hand. Just look at all changes found in version control history, having the guarantee that the platform always matches what is described there.
3. Auto-Pull
Using GitOps means that you employ software agents that are always running in the cluster, automatically pulling Git state at regular intervals and checking it against the live cluster state. This way you always know if the version in Git is the same as the live state or not.
4. Loop the Loop
Operating in a close loop ensuring the desired state of a system matches the declared state. This is one of the most important features because it provides feedback allowing you and your team to better control your operations and workflow.
When used with your repository, GitOps software agents can perform various functions, ensuring self-healing. Agents automate fixes in case there’s an outage, perform QA processes for your workflow, and protect against human error or manual intervention.
GitOps Use Cases
GitOps for Developers
Developers can use GitOps to streamline their workflow, as it leverages Git’s version control capabilities to manage infrastructure changes. When a developer pushes a change to the Git repository, the GitOps process automatically initiates a deployment pipeline. This eliminates the need for developers to manually oversee each deployment, making it faster and reducing the risk of errors. Plus, the use of pull requests for changes enables a peer-review process that improves quality control.
GitOps for Platform Engineers
Platform engineers can take advantage of GitOps to enhance infrastructure management. By storing infrastructure configurations in a Git repository, they can easily track changes, rollback if necessary, and achieve consistency across multiple environments. GitOps also provides a system for automating deployments and updates, saving time and reducing manual effort.
GitOps for Kubernetes and Cloud Native
For teams deploying containerized applications on Kubernetes or other cloud-native environments, GitOps provides an effective way to manage deployments. The declarative nature of GitOps aligns well with Kubernetes’ architecture, allowing it to automatically reconcile any differences between the desired state described in Git and the actual state of the cluster. GitOps also helps manage complex, multi-cluster and multi-environment Kubernetes deployments.
GitOps for Teams Adopting Continuous Deployment
Teams adopting continuous deployment practices can significantly benefit from GitOps. It helps automate the deployment process, ensuring every commit triggers a new, reliable deployment. The ability to track changes, revert, and fork allows for quick rollback in case of any issues, leading to higher availability of services. Additionally, with GitOps, every step in the deployment pipeline is transparent and auditable, improving visibility and control.
Learn more in our detailed guide to GitOps benefits.
Getting Hooked on GitOps
It’s easy to get started with GitOps, because is a model developed on pre-existing practices:
- Git workflows – GitOps uses Git repositories as the change mechanism for application and infrastructure updates. Any change to an application or environment is represented as a change in a Git repository – providing a full audit trail and the ability to roll back to a previous version at any time.
- CD pipelines – with GitOps you have a clear separation between CI and CD. Your CI pipelines stay as is and their end result is still an artifact and/or container that is stored in a binary repository. Once the artifact is created, GitOps takes over and applies it to the target cluster. Instead of triggering the CI/CD with a commit in the source code, with GitOps the trigger is a commit to the configuration repository.
- Infrastructure as Code (IaC) – GitOps uses a Git repository as the single source of truth for your environments. Infrastructure as code (IaC) means storing all infrastructure configuration as code in a version control system. IaC systems use declarative configuration, meaning you only state what you need deployed, not how.
- Tracking and observability – observability, a property of a system which allows it to be easily observed, is an important GitOps concept. It lets you ensure that the desired state described in declarative configuration, and actual current state, are the same.
- Immutable servers (containerized infrastructure) – GitOps is often used to operate Kubernetes application development. It lets you build development pipelines, continuously deploy applications, provision clusters, and deploy on Kubernetes or container registries. However, GitOps can also be used in other infrastructure environments.
This means you can adopt GitOps today with your existing tools while still following the DevOps mindset.
Learn more in our detailed guide to GitOps branching strategies
VP of Open Source, Octopus Deploy
Dan is a seasoned leader in the tech industry with a strong focus on open source initiatives. Currently serving as VP of Open Source at Octopus Deploy, contributing as an Argo maintainer, co-creator of Open GitOps, and leveraging extensive experience as a co-founder of Codefresh, now part of Octopus Deploy.
TIPS FROM THE EXPERT
In my experience, here are tips that can help you better adapt to GitOps:
- Use declarative tooling: Emphasize the use of declarative tools and configurations. Tools like Helm for Kubernetes applications and Terraform for infrastructure as code can help ensure that your desired state is clearly defined and reproducible.
- Adopt a multi-repo strategy: Consider using multiple repositories for different components of your infrastructure and applications. This can help manage complexity, improve security, and enable more fine-grained control over changes.
- Monitor drift and reconcile frequently: Use tools like Flux or Argo CD to continuously monitor for drift between the desired state in Git and the actual state in your environment. Ensure frequent reconciliation to maintain consistency and detect issues early.
- Integrate with CI/CD tools: Integrate GitOps workflows with your CI/CD tools to automate the entire pipeline from code commit to deployment. This ensures that all changes go through the same rigorous testing and approval processes before being applied to production.
- Automate policy enforcement: Use tools like Open Policy Agent (OPA) or Kyverno to enforce policies directly from your Git repositories. This ensures compliance and governance across your infrastructure and applications.
GitOps at Your Service
Most existing processes for infrastructure configuration management face challenges like failed deployments, poor infrastructure design, server outages, etc. These challenges can’t be solved by only the traditional approach of IaC (Infrastructure as Code).
GitOps can help you with these common challenges:
- Lack of configuration consistency – even with centralized configuration, the actual configuration and a declared one can alter and change with manual processes.
- No idea how an application is running or where – this sometimes entails a full-scale investigation to determine state and configuration of infrastructure.
- Failed deployments relying on a disaster recovery strategy – disaster recovery requires a strict process, typically with manual processes, which isn’t always reliable. This results in unplanned downtime that affects customers and hurts developer productivity.
- Missing documentation – itcan be difficult to figure out how some applications were built and who managed them.
- Relying on previous state for success – this can be unpredictable, which affects the stability and state of a project.
GitOps provides the following key benefits:
- Increased developer productivity – GitOps lets developers use familiar tools like Git to manage updates and features, without having to deal with containers or know the internal of Kubernetes.
- Minimize failed deployments and rapidly recover from downtime – Git provides the ability to revert, rollback, and fork, enabling easy rollback if anything goes wrong in a production environment. Git provides a single source of truth, reducing time to recovery to minutes or seconds.
- Faster time to ship features in production – GitOps lets developers continuously deploy feature updates to Kubernetes clusters. Deployments become instant, consistent, and reliable.
- Built-in auditing history – when you use Git to manage Kubernetes clusters, you gain a complete audit log of all changes to the cluster originating outside of Kubernetes with a full audit trail of who changed what. This enables consistent operations and supports compliance.
- Greater visibility on the lifecycle of developed features – GitOps provides a consistent model for infrastructure and application changes across the organization. Development processes are visible and fully reproducible through Git.
- Improved security – Git provides strong guarantees on correctness and security, with strong cryptography used to manage changes and verify authorship. This dramatically improves security and means that CI systems are no longer a threat vector for production systems.
These benefits are applicable to both operators/service engineers, as well as developers who need a more robust operational solution for application development and infrastructure automation.
Ouch…5 Things that can Make GitOps Hard
While GitOps has compelling benefits, it’s not always smooth sailing. Understanding the common challenges can help you make your journey smoother. Here are the main drawbacks of GitOps and what you can do about them.
1. Automating Git Commits May Create Conflicts
It is important to update applications in the CD automatically to accommodate frequent updates. The CI process must create pull requests when it requests to release an update to the testing environment. However, Git supports manual conflict resolution and editing, not automatic updates, so multiple CI processes can cause conflicts when writing to the same repository.
These conflicts don’t result from specific files but from the lack of synchronization between different processes cloning the same repo. You can mitigate this issue by using multiple repositories – for example, one for each namespace.
2. Too Many Git Repositories
The number of GitOps repositories typically increases with each new environment or application, increasing the repo management burden. Every repository must have the same access permissions and connect to a sync agent on various clusters. Provisioning these repositories can take up a large chunk of the development time.
To mitigate this issue, use fewer Git repositories (e.g., one per cluster). However, this approach means each repository must do more regarding pull-request and access control management.
3. Limited Visibility
GitOps provides visibility into everything within an environment, with all intended state data stored in Git in plain text. However, this only works with simpler setups (i.e., fewer repositories containing manageable numbers of config files). Enterprise environments often have many GitOps repositories and config files, making it impractical to comb through plain text files.
4. Doesn’t Centrally Manage Secrets
A complex enterprise environment requires a solution to manage secrets outside the standard CI/CD process. It is important to audit secrets (i.e., keys, passwords), so storing them in a secure, centralized data store makes sense. While GitOps won’t prevent you from managing secrets centrally, it doesn’t facilitate this either.
A Git repository is not suitable for storing secrets requiring encryption and decryption. Git history will remember the secrets forever.
5. Auditing May Be Insufficient
Git repositories are useful for auditing processes because they retain a full history of all state changes. However, it is harder to answer other questions with a Git repo, such as the number of times an application was deployed. These tasks require walking through the Git history or searching through text files.
GitOps vs. DevOps
DevOps (development + operations) is a collaborative approach to performing tasks by development and operations teams. While DevOps is a general philosophy of how teams should work (i.e., best practices applicable across an organization’s processes), GitOps is a way to effectively implement DevOps. It is a method that depends on a specific tool for source control (Git) and is more goal-oriented.
GitOps uses Git to verify and deploy infrastructure changes automatically. It handles operations with a declarative approach – DevOps involves prescriptive and declarative approaches. DevOps can apply to any infrastructure, not just containers.
DevOps focuses on operations and deployment, which is useful for organizations making slow application changes and handling frequent infrastructure issues and changes. It is suitable for software development and deployment and can help manage infrastructure as code (IaC).
GitOps is becoming a common workflow for managing containerized applications and cloud infrastructure because it helps implement CI/CD pipelines on Kubernetes. It uses Git (GitLab or GitHub) to redefine CI/CD and serve as an immutable, central state declaration.
GitOps Example: a Sample GitOps Workflow
Developers already use Git for the source code of the application. GitOps extends this practice to an application’s configuration, infrastructure, and operational procedures.
All changes (both on the application and on infrastructure level) are automated through the source control system that provides a single source of truth. A GitOps controller is responsible for matching Git state with actual system state in a bidirectional manner – match the cluster to Git, but also detect changes to the cluster that are not part of a Git commit.
The basic characteristics of GitOps are:
- Git is the single source of truth
- Declarative systems and configuration
- Immutable deployments
- A central auditable changelog
Basic GitOps workflow for Kubernetes
- Developer commits changes to code in the Git repository that holds application source code. A container image is created and pushed to the container registry.
Each application repository contains the source code that developers write to build an application. Developers can still work the same way as they worked before as far as the application logic is concerned. A commit always results in an artifact (usually a container image).
The container registry is a collection of repositories that stores container images. A container image includes everything that the application needs to run.
- There is a second Git repository that holds deployment configuration for the application.
In the case of Kubernetes, this includes the manifests that describe the deployment and other associated changes. Configuration files are written in a YAML format that includes the definitions of how to map external ports, directories, services, environment variables, and configuration options for the image. Essentially, the desired state of an application.
- Any change to the Git repository is monitored by the GitOps agent. Once a change is detected, an application (or infrastructure) deployment takes place.
Requesting a change to the desired state of the cluster initiates automatic syncing with Argo CD or whatever tool you use to automate the state change.
This step monitors the application and compares it to the current, live state against the desired target state, specified in the Git repository. The sync works both ways. If a change happens to a cluster (that is different from what is described in Git), it can be discarded or applied back to Git.
- The image is rolled out to the staging or production environment.
You can handle multiple environments or clusters with GitOps. A common pattern would be to deploy to a staging environment first for running load testing or integration testing. This is an extra layer of reliability, making sure there are no issues with the deployment prior to Production.
- When the deployment completes, developers can test the application to verify success and see that cluster state is the same as the Git state.
This workflow is an example of how GitOps works. It’s a paradigm that can be applied to any cloud-native environment to manage infrastructure provisioning as well as software deployment.
Learn more in our detailed guide to using Helm with GitOps
Using Codefresh to Adopt GitOps
Codefresh lets your team easily adopt GitOps to achieve sustainable and extensible DevOps.
Built on GitOps for traceability and reliable management
Codefresh is the only enterprise DevOps solution that operates completely with GitOps from the ground up. Using the CLI or GUI in Codefresh generally ends with a git commit. Whether that’s installing the platform, creating a pipeline, or deploying new software. The CLI/GUI simply acts as extended interfaces of version control. A change to the desired state of the software supply chain will automatically be applied to the actual state.
In Codefresh, every change and configuration goes directly into git. This makes git a common language for both engineering and operations and provides a full change history. Simplified change management, and permission management in the platform scope each user to their permissions in git and provide a full changelog for auditability of every change.
Codefresh Delivery Pipelines form the backbone of the software supply chain
You can’t achieve continuous delivery without first solving continuous integration. A Codefresh Delivery Pipeline is the combination of a workflow along with the events that trigger it.
We’ve added a pipeline creation wizard that will create all the component configurations so you can spend less time with YAML and more time getting work done. The pipeline creation wizard ends with committing the configuration to git and allowing the built-in Argo CD instance to deploy them to the relevant runtimes.
The Delivery Pipeline model also allows you to create a single reusable pipeline that operates based on the context of its triggers. Pipelines can easily work as templates to be reused by thousands of different applications. DevOps teams can build once and use it everywhere.
Current Argo users will especially appreciate how much simpler it is to create DevOps pipelines because sensors and workflow creation are merged into a single flow. The visibility in Codefresh also makes debugging issues with events and workflows much easier as they are associated and logs are collected into the main view of each execution.
Codefresh GitOps Certification for Argo
Join the nearly 20,000 students who have transformed their DevOps skills with Codefresh’s industry-leading GitOps Certification. With two courses – Fundamentals and At Scale, you’ll learn the theory of GitOps and also apply all of these practices in your own application using the Argo project family.
See Additional Guides on Key DevOps Topics
Together with our content partners, we have authored in-depth guides on several other topics that can also be useful as you explore the world of DevOps.
Authored by Codefresh
Authored by Faddom
The World’s Most Modern CI/CD Platform
A next generation CI/CD platform designed for cloud-native applications, offering dynamic builds, progressive delivery, and much more.
Check It Out