What Is GitOps? How Git Can Make DevOps Even Better

What is GitOps?

GitOps is a set of best practices applied from the beginning of the development workflow, all the way to deployment.

Developers already use Git for the source code of the application – GitOps extends this practice to an application’s configuration, infrastructure, and operational procedures. All changes to applications and infrastructure are described in a source control system and automatically synchronized with the live environment.

It is important to note that GitOps is not the next thing after DevOps or even DevOps 2.0.

GitOps is a set of deployment practices while DevOps is a paradigm or better yet, a mindset. Their shared principles make it easier for teams to adopt the GitOps workflow for existing DevOps techniques.

The 4 Commandments of GitOps

Here are the four things GitOps practitioners swear by. You absolutely need them to do GitOps right.

1. Declarative Everything

By leveraging a declarative format, you can describe how your infrastructure works as well as the applications that are running on top of it. Doing this enables traceability of changes made to any environment and enables rollback, recoverability, and self-healing attributes using any source control system. Move away for imperative adhoc scripts to declarative configuration in all levels (app and infra).

2. V for Versioning

Declarative descriptions stored within the repository supports immutability, versioning, and versioning history. For example, using Git for the declarations mentioned above, allows you a single place from which everything for your application is derived and driven. This allows you to easily pinpoint any changes made at any time. Don’t try to understand what is different between two environments by hand. Just look at all changes found in version control history, having the guarantee that the platform always matches what is described there.

3. Auto-Pull

Using GitOps means that you employ software agents that are always running in the cluster, automatically pulling Git state at regular intervals and checking it against the live cluster state. This way you always know if the version in Git is the same as the live state or not.

4. Loop the Loop

Operating in a close loop ensuring the desired state of a system matches the declared state. This is one of the most important features because it provides feedback allowing you and your team to better control your operations and workflow.

When used with your repository, GitOps software agents can perform various functions, ensuring self-healing. Agents automate fixes in case there’s an outage, perform QA processes for your workflow, and protect against human error or a manual intervention.

Getting Hooked on GitOps

It’s easy to get started with GitOps, because is a model developed on pre-existing practices:

• Git workflows – GitOps uses Git repositories as the change mechanism for application and infrastructure updates. Any change to an application or environment is represented as a change in a Git repository – providing a full audit trail and the ability to roll back to a previous version at any time.
• CD pipelines – with GitOps you have a clear separation between CI and CD. Your CI pipelines stay as is and their end result is still an artifact and/or container that is stored in a binary repository. Once the artifact is created, GitOps takes over and applies it to the target cluster. Instead of triggering the CI/CD with a commit in the source code, with GitOps the trigger is a commit to the configuration repository.
• Infrastructure as Code (IaC) – GitOps uses a Git repository as the single source of truth for your environments. Infrastructure as code (IaC) means storing all infrastructure configuration as code in a version control system. IaC systems use declarative configuration, meaning you only state what you need deployed, not how.
• Tracking and observability – observability, a property of a system which allows it to be easily observed, is an important GitOps concept. It lets you ensure that the desired state described in declarative configuration, and actual current state, are the same.
• Immutable servers (containerized infrastructure) – GitOps is often used to operate Kubernetes application development. It lets you build development pipelines, continuously deploy applications, provision clusters, and deploy on Kubernetes or container registries. However, GitOps can also be used in other infrastructure environments.

This means you can adopt GitOps today with your existing tools while still following the DevOps mindset.

GitOps at Your Service

Most existing processes for infrastructure configuration management face challenges like failed deployments, poor infrastructure design, server outages, etc. These challenges can’t be solved by only the traditional approach of IaC (Infrastructure as Code). 

GitOps can help you with these common challenges:

• Lack of configuration consistency – even with centralized configuration, the actual configuration and a declared one can alter and change with manual processes.
• No idea how an application is running or where – this sometimes entails a full-scale investigation to determine state and configuration of infrastructure.
• Failed deployments relying on a disaster recovery strategy – disaster recovery requires a strict process, typically with manual processes, which isn’t always reliable. This results in unplanned downtime that affects customers and hurts developer productivity.
• Missing documentation – itcan be difficult to figure out how some applications were built and who managed them.
• Relying on previous state for success – this can be unpredictable, which affects the stability and state of a project.

GitOps provides the following key benefits:

• Increased developer productivity  – GitOps lets developers use familiar tools like Git to manage updates and features, without having to deal with containers or know the internal of Kubernetes.
• Minimize failed deployments and rapidly recover from downtime – Git provides the ability to revert, rollback, and fork, enabling easy rollback if anything goes wrong in a production environment. Git provides a single source of truth, reducing time to recovery to minutes or seconds. 
• Faster time to ship features in production – GitOps lets developers continuously deploy feature updates to Kubernetes clusters. Deployments become instant, consistent, and reliable. 
• Built-in auditing history – when you use Git to manage Kubernetes clusters, you gain a complete audit log of all changes to the cluster originating outside of Kubernetes with a full audit trail of who changed what. This enables consistent operations and supports compliance.
• Greater visibility on the lifecycle of developed features – GitOps provides a consistent model for infrastructure and application changes across the organization. Development processes are visible and fully reproducible through Git.  
• Improved security – Git provides strong guarantees on correctness and security, with strong cryptography used to manage changes and verify authorship. This dramatically improves security and means that CI systems are no longer a threat vector for production systems.

These benefits are applicable to both operators/service engineers, as well as developers who need a more robust operational solution for application development and infrastructure automation.

Ouch...5 Things that can Make GitOps Hard

While GitOps has compelling benefits, it’s not always smooth sailing. Understanding the common challenges can help you make your journey smoother. Here are the main drawbacks of GitOps and what you can do about them.

1. Automating Git Commits May Create Conflicts

It is important to update applications in the CD automatically to accommodate frequent updates. The CI process must create pull requests when it requests to release an update to the testing environment. However, Git supports manual conflict resolution and editing, not automatic updates, so multiple CI processes can cause conflicts when writing to the same repository.

These conflicts don’t result from specific files but from the lack of synchronization between different processes cloning the same repo. You can mitigate this issue by using multiple repositories – for example, one for each namespace.

2. Too Many Git Repositories

The number of GitOps repositories typically increases with each new environment or application, increasing the repo management burden. Every repository must have the same access permissions and connect to a sync agent on various clusters. Provisioning these repositories can take up a large chunk of the development time.

To mitigate this issue, use fewer Git repositories (e.g., one per cluster). However, this approach means each repository must do more regarding pull-request and access control management.

3. Limited Visibility

GitOps provides visibility into everything within an environment, with all intended state data stored in Git in plain text. However, this only works with simpler setups (i.e., fewer repositories containing manageable numbers of config files). Enterprise environments often have many GitOps repositories and config files, making it impractical to comb through plain text files.

4. Doesn’t Centrally Manage Secrets

A complex enterprise environment requires a solution to manage secrets outside the standard CI/CD process. It is important to audit secrets (i.e., keys, passwords), so storing them in a secure, centralized data store makes sense. While GitOps won’t prevent you from managing secrets centrally, it doesn’t facilitate this either.

A Git repository is not suitable for storing secrets requiring encryption and decryption. Git history will remember the secrets forever.

5. Auditing May Be Insufficient

Git repositories are useful for auditing processes because they retain a full history of all state changes. However, it is harder to answer other questions with a Git repo, such as the number of times an application was deployed. These tasks require walking through the Git history or searching through text files.

GitOps vs. DevOps

DevOps (development + operations) is a collaborative approach to performing tasks by development and operations teams. While DevOps is a general philosophy of how teams should work (i.e., best practices applicable across an organization’s processes), GitOps is a way to effectively implement DevOps. It is a method that depends on a specific tool for source control (Git) and is more goal-oriented.

GitOps uses Git to verify and deploy infrastructure changes automatically. It handles operations with a declarative approach – DevOps involves prescriptive and declarative approaches. DevOps can apply to any infrastructure, not just containers.

DevOps focuses on operations and deployment, which is useful for organizations making slow application changes and handling frequent infrastructure issues and changes. It is suitable for software development and deployment and can help manage infrastructure as code (IaC).

GitOps is becoming a common workflow for managing containerized applications and cloud infrastructure because it helps implement CI/CD pipelines on Kubernetes. It uses Git (GitLab or GitHub) to redefine CI/CD and serve as an immutable, central state declaration.

GitOps Example: a Sample GitOps Workflow

Developers already use Git for the source code of the application. GitOps extends this practice to an application’s configuration, infrastructure, and operational procedures.

All changes (both on the application and on infrastructure level) are automated through the source control system that provides a single source of truth. A GitOps controller is responsible for matching Git state with actual system state in a bidirectional manner – match the cluster to Git, but also detect changes to the cluster that are not part of a Git commit.

The basic characteristics of GitOps are:

• Git is the single source of truth
• Declarative systems and configuration
• Immutable deployments
• A central auditable changelog

Basic GitOps workflow for Kubernetes

1. Developer commits changes to code in the Git repository that holds application source code. A container image is created and pushed to the container registry.

Each application repository contains the source code that developers write to build an application. Developers can still work the same way as they worked before as far as the application logic is concerned. A commit always results in an artifact (usually a container image).

The container registry is a collection of repositories that stores container images. A container image includes everything that the application needs to run.

2. There is a second Git repository that holds deployment configuration for the application.

In the case of Kubernetes, this includes the manifests that describe the deployment and other associated changes. Configuration files are written in a YAML format that includes the definitions of how to map external ports, directories, services, environment variables, and configuration options for the image. Essentially, the desired state of an application.

3. Any change to the Git repository is monitored by the GitOps agent. Once a change is detected, an application (or infrastructure) deployment takes place.

Requesting a change to the desired state of the cluster initiates automatic syncing with Argo CD or whatever tool you use to automate the state change. 

This step monitors the application and compares it to the current, live state against the desired target state, specified in the Git repository. The sync works both ways. If a change happens to a cluster (that is different from what is described in Git), it can be discarded or applied back to Git.

4. The image is rolled out to the staging or production environment.

You can handle multiple environments or clusters with GitOps. A common pattern would be to deploy to a staging environment first for running load testing or integration testing. This is an extra layer of reliability, making sure there are no issues with the deployment prior to Production.

5. When the deployment completes, developers can test the application to verify success and see that cluster state is the same as the Git state.

This workflow is an example of how GitOps works. It’s a paradigm that can be applied to any cloud-native environment to manage infrastructure provisioning as well as software deployment.

Using Codefresh to Adopt GitOps

Codefresh lets your team easily adopt GitOps to achieve sustainable and extensible DevOps.

Built on GitOps for traceability and reliable management

Codefresh is the only enterprise DevOps solution that operates completely with GitOps from the ground up. Using the CLI or GUI in Codefresh generally ends with a git commit. Whether that’s installing the platform, creating a pipeline, or deploying new software. The CLI/GUI simply acts as extended interfaces of version control. A change to the desired state of the software supply chain will automatically be applied to the actual state.

In Codefresh, every change and configuration goes directly into git. This makes git a common language for both engineering and operations and provides a full change history. Simplified change management, and permission management in the platform scope each user to their permissions in git and provide a full changelog for auditability of every change.

Codefresh Delivery Pipelines form the backbone of the software supply chain

You can’t achieve continuous delivery without first solving continuous integration. A Codefresh Delivery Pipeline is the combination of a workflow along with the events that trigger it. 

We’ve added a pipeline creation wizard that will create all the component configurations so you can spend less time with YAML and more time getting work done. The pipeline creation wizard ends with committing the configuration to git and allowing the built-in Argo CD instance to deploy them to the relevant runtimes.

The Delivery Pipeline model also allows you to create a single reusable pipeline that operates based on the context of its triggers. Pipelines can easily work as templates to be reused by thousands of different applications. DevOps teams can build once and use everywhere. 

Current Argo users will especially appreciate how much simpler it is to create DevOps pipelines because sensors and workflow creation are merged into a single flow. The visibility in Codefresh  also makes debugging issues with events and workflows much easier as they are associated and logs collected into the main view of each execution.