One of the major components of the Codefresh Software Delivery Platform is the Workflows capability that allows you to define any kind of software process for creating artifacts, running unit tests, running security scans, and all other actions that are typically used in Continuous Integration (CI).
At first glance, Codefresh Workflows might look like the typical pipelines that you would find in any popular CI product but if you look under the hood you will realize looks can be deceiving. Codefresh workflows redefine the way pipelines are created by bringing GitOps into the mix and for the first time ever adopting a Git-based process instead of the usual ClickOps.
Here is an example of a Codefresh workflow as it is presented in the graphical user interface.
The tree representation shows you an overview of the workflow along with its major components/steps and how they communicate with each other.
The real power of Codefresh workflows however, becomes evident when we explain how you create and manage them (compared to the usual approaches from traditional CI systems).
Codefresh workflows are powered by Argo Workflows and Argo Events
The Codefresh platform is powered by the open source Argo projects and workflows are no exception. The engine that is powering Codefresh workflows is the popular Argo Workflows project accompanied with Argo Events.
Traditionally Codefresh has been using a proprietary engine for modeling CI pipelines which has served us well but has also required upkeep and maintenance by the core Codefresh team. In our new platform we now use Argo Workflows/Events and are fully adopting the open source development model. This way Codefresh can move towards a standardized and open workflow runtime while at the same time giving back all our contributions to the community for other organizations to benefit in the same way.
On a technical level adopting Argo Workflows has the following advantages:
- We now use a mature and battle-tested runtime for all pipelines
- We gain the flexibility of the Argo engine and its Kubernetes native capabilities such as being able to run each step in its own Kubernetes pod
- We can offer to all customers a set of reusable steps in the form of Workflow templates
- We can reuse all existing Artifact integrations instead of spending extra effort to create our own.
The non-technical advantages are even more important:
- Organizations already familiar with Argo Workflows can adopt Codefresh workflows with minimal effort
- We can now move at a faster pace when adding features than when we had only the Codefresh core team maintaining a proprietary workflow engine
- Codefresh becomes both a consumer of Argo Workflows and a contributor to them (We give all our modifications and enhancements back to the open source community)
- The Argo Workflows project as a whole is strengthened by a major contributor that also uses it internally (we eat our own dog food by using the Codefresh platform to develop the Codefresh platform)
It is important to note that apart from individual features and enhancements, one major contribution to the Argo ecosystem is the introduction and hosting of the Codefresh Hub for Argo Workflows.
The Hub allows you to find curated Argo templates and use them in your workflows, while at the same time you can publish your own template giving back to the community. The Codefresh Argo Hub enables everybody to share and reuse Argo Workflow templates in a way that was never possible before.
Integrated installation and management of your workflow runtime
Argo Workflows is a Kubernetes native application, and thus can be easily installed on any compliant Kubernetes cluster. The initial installation is not a big challenge if you only have a single instance of Argo Workflows. However, most organizations wish to employ multiple clusters/regions for running workflows and keeping track of multiple Workflow installations can pose major challenges such:
- It would make sense to use ArgoCD to manage Argo Workflows but that presents the question on how to manage ArgoCD itself
- Argo Workflows is rarely used on its own. It requires several artifact repositories and usually it is also coupled with Argo Events in order to get the full power of workflows. This means that a workflow installation is a set of applications that must be handled together
- Using Argo Workflows with multiple instances split on different clusters is not a straightforward operation and currently lacks a central control plane
The Codefresh platform solves all these problems by introducing the concept of a Runtime. The Codefresh runtime is a collection of all 4 Argo projects (including Argo Workflows and Argo Events) that is installed and handled as a single unit. The Codefresh team is going to great lengths to make sure that all Argo components are of proper versions, in addition to passing several common enterprise requirements such as security scans.
The key point here is that Codefresh is enforcing GitOps all the way through. When a runtime is created all of its components are installed/managed by ArgoCD. ArgoCD also manages itself making the Runtime a self-contained unit. For more information on how this works see our open source project Argo Autopilot.
In addition to the initial installation, the Codefresh runtime can be easily upgraded in a safe manner by following the GitOps principles.
Handling Argo Events and Argo Workflows as a single unit is very important as we will see in the next section, but the main highlight of using runtimes is the fact that you can define and manage multiple runtimes from a single Codefresh dashboard.
This makes the Codefresh Delivery platform a central control plane for all your Argo Workflow installations making it very easy to adopt the Argo projects in all multi-cluster scenarios.
It is also important to note that this capability is present for all Argo projects when using Codefresh as a control plane including ArgoCD. A single ArgoCD instance can handle multiple clusters, but if you want to have multiple ArgoCD instances you still need to look at the respective number of dashboards. With Codefresh, this is not needed anymore.
So to summarize, the introduction of the Codefresh runtime has the following benefits:
- A runtime collects all 4 Argo projects in a single cohesive unit
- The runtime is self-contained as it is used ArgoCD to manage all of its components including ArgoCD itself
- All version components are guaranteed to work together in the most optimal way
- Specifically for Argo Workflows an associated Argo Events instance is installed alongside
- All components are managed using GitOps instead of ad-hoc installation methods
- Codefresh works as a single control plane making multi-cluster and multi-environment installation and upgrades very easy
Every change in a workflow is backed by a Git commit
The Codefresh Software Delivery Platform is fully embracing GitOps. And while most people might employ GitOps only as a way to deploy applications, we take it one step further and use GitOps everywhere, including workflow management.
Remember that each Runtime defined in the previous section also comes with its own ArgoCD instance and associated Git integrations. We can reuse these integrations to manage workflows itself.
This is a major differentiation between plain Argo Workflows and Codefresh workflows. Argo Workflows can be stored anywhere (or nowhere) while Codefresh workflows is an opinionated solution that enforces best practices via GitOps.
The full adoption of GitOps becomes evident from the time you want to create a new workflow. The very first question is which Git repository will be used for its storage:
After the initial creation, any further changes you make are always stored in Git. Anything that you modify is backed by a Git commit.
But simply using Git for storage is not GitOps. We instead monitor workflows at all times and always check their Git state against their actual state. You can always see what is happening in the “manifests” view of a Codefresh workflow.
The 3 tabs clearly show what is happening for each file:
- “Git state” shows the file as stored in Git
- “Desired state” shows the expected runtime state (usually has extra annotation and labels)
- “Live state” shows what is actually in your Kubernetes cluster.
Full adoption of GitOps for workflows and not just applications is a unique feature of the Codefresh platform and we believe that this is the way GitOps should be employed when it comes to workflows.
As a final note, remember that the Codefresh runtime includes all 4 Argo Components. This means that the same principles apply to all other components that are needed for the whole workflow functionality.
In the case of an Argo Workflow we also store the same way Sensors and Trigger (Argo Events components) that take part in the definition of a workflow.
This solves a very common challenge of handling multiple Argo projects together. As we already explained, Argo Workflows is rarely used on its own. Most non-trivial installations will also require Argo Events to fully unlock the power of workflows.
With the Codefresh runtime everything is packaged in a cohesive manner that still follows all the best practices that come from GitOps
Fine-Tuned workflows specifically for CI
Argo Workflows is a generic Workflow solution with several use cases. Two of the most popular use cases are machine learning and Continuous Integration. Having a flexible solution is an advantage for organizations that wish to use Argo Workflows in different scenarios, but can be a challenge for organizations that need a turn-key solution that works out of the box.
At Codefresh our focus is on Continuous Integration and Deployment. This means that the version of Argo Workflows included in the Codefresh Delivery Platform is already fine tuned specifically for Software Delivery.
This means that no customization is required in order to get a working pipeline. Organizations that adopt the Codefresh platform can start building and deploying software with zero effort while still taking all the advantages that Argo Workflows offer under the hood.
The Codefresh platform offers several enhancements on top of vanilla Argo Workflows but the most important ones are :
- A complete trigger system with all the predefined events that users of a CI/CD platform expect (i.e. trigger on Git push, PR open, branch delete etc)
- A set of predefined pipeline steps that are commonly used in CI/CD pipelines (security scanning, image building, unit tests, etc)
- A build view specifically for CI/CD pipelines.
Here you can see again the importance of having a common runtime that bundles Argo Workflows plus Argo Events. If you use Argo Workflows on your own, it is up to you to create the appropriate triggers and sensors that you need for your CI/CD pipelines. With Codefresh you get all this out of the box.
As a quick example, Codefresh offers you all the Git events that you might need for your pipeline in a friendly graphical interface. Of course under the hood everything is stored in Git in the forms of sensors and triggers.
A smart filtering system is also offered for more advanced workflows. This comes in a dynamic UI component that allows you to chain multiple filters with different conditions. Some examples are:
- Trigger a workflow only when somebody opens a Pull request that is targeted at “main”
- Trigger a workflow only when a branch named “feature-*” is created
- Trigger a workflow only when a Git tag is created for repository X
All these are very common scenarios that users expect from traditional CI/CD platforms. With Codefresh workflows you have full access to all predefined filters and can also create your own with minimal effort.
The second important feature is the integration of the Codefresh Hub for Argo that contains Workflow templates geared towards CI/CD pipelines. You can already see some examples such as :
The availability of all these templates makes pipeline creation very easy, as anybody can simply connect existing steps in a lego-like manner to create a complete pipeline instead of manually writing workflow templates from scratch.
Finally the build view of the Codefresh platform brings together the output of Argo Workflows with all the useful information that a CI pipeline needs (such as the committer and the Git commit that initiated the pipeline) along with search filters for handling a large number of builds.
Full traceability between development and deployment
The last important point regarding Codefresh workflows is the way they are connected to the rest of the Codefresh platform.
Since the Codefresh Software Delivery Platform touches all aspects of software development you have a single control plane for everything that starts from a Git commit all the way to a production deployment.
This means that the context of Codefresh workflows is augmented with the other parts of the platform and more specifically the applications affected by a pipeline, which Kubernetes changes were deployed, which container images were produced and so on.
This all comes down to the central GitOps dashboard that connects everything within the Codefresh platform
This dashboard aggregates all information from all lifecycle stages:
- Who initiated a change
- What was the Git commit
- What features (i.e. Jira tickets) are included
- Which Pull request took part in the change
- Which pipeline was used for artifact creation
- What was a result of security scans/unit tests/lint checks etc
- Which container images were produced and pushed
- Which Kubernetes applications were affected
- Which individual components were updated
You cannot get all this information with plain Argo Workflows. This dashboard is the testament to the power of the Codefresh platform as it brings you unprecedented visibility to your applications by using a single view of your whole process. You don’t need to hunt down individual dashboards and manually correlate information between builds/images/commits/applications etc.
Get access to Codefresh Workflows now
We have seen the basic building blocks of Codefresh workflows and how they enhance the traditional Argo Workflows.
Codefresh is adopting GitOps in all areas of the platform and allows you to manage your workflows like any other application. At the same time it fine-tunes your workflows specifically for handling all CI/CD use cases that are needed in an organization.
Codefresh Workflows are only one component of the Codefresh Software Delivery platform. Starting today with additional features rolling out over the coming weeks, CSDP is available free for small teams and with a free trial for larger teams. Get started at https://codefresh.io/codefresh-signup/
