What Is CI/CD?
Continuous Integration and Continuous Delivery/Deployment (CI/CD) is a software engineering practice that automates application build, testing, and deployment. CI/CD practices form the backbone of modern DevOps processes.
A CI/CD pipeline compiles incremental code changes made by developers and packages them into software artifacts. Automated testing verifies the integrity and functionality of the software, and automated deployment services make it immediately available to end users. The goal is to enable early detection of defects, increase productivity, and shorten release cycles.
This process contrasts with the traditional approach to software development—consolidating multiple small software updates into one large release, thoroughly testing it, and only then deploying it. CI/CD pipelines support the agile concept of development in small iterations, enabling teams to deliver value to customers faster, and create a rapid feedback loop for developers.
What Are the Differences Between Continuous Integration, Continuous Delivery, and Continuous Deployment?
In the traditional software development process, multiple developers produce code, and only towards the end of a release do they consolidate their work. This caused many bugs and issues, which could only be identified and resolved after a long testing phase. Until all those issues were resolved, the software could not be released. This hurt software quality, and meant that teams could typically only release new versions once or twice a year.
Continuous Integration (CI) was designed to solve this problem and support agile development processes. CI means that any changes developers make to their code are immediately integrated into the master branch of the software project. The CI system automatically runs tests to catch quality issues, and developers get quick feedback and can fix issues immediately. Developers often commit to the master branch or work on a short-lived feature branch, and a feature is not considered complete until it is integrated with other code changes in the master branch.
In a CI process, a build server is responsible for taking new code changes, running automated tests using multiple tools, integrating the code into the master branch, and generating a build—a new version of software artifacts needed to deploy the software.
CI greatly improves the quality and speed of software development. Teams can create more features that provide value to users, and many organizations now release software every week, every day, or multiple times a day.
Traditionally, deploying new software versions has been a large, complex and risky task. After the new version was tested, the operations team was tasked with deploying it into production. Depending on the size of the software it could take hours, days or weeks, requires detailed checklists and many manual steps, and special expertise. Deployments often failed and required developer workarounds or urgent assistance.
There are many problems with this traditional approach—it is stressful for the team, expensive and risky for the organization, and causes bugs and downtime in production environments.
Continuous Delivery (CD, also known as CDel) aims to solve these problems through automation. The CD approach allows teams to package software and deploy it to production environments with the push of a button. The basic principle of CD is that any change to a software project can be deployed to a production environment immediately, without any special effort.
After the CI system consolidates the new changes and creates a new build, the CD system packages the new version, deploys it to a test environment, automatically evaluates its behavior, and pushes it to the production environment. This last step can be manually approved, but no manual action is required to deploy the new version to production.
Implementing CD requires automating the entire software development lifecycle, including build, test, environment setup, and deployment. All artifacts must reside in a source code repository, and an automated mechanism is required to create and update the environment.
A true CD pipeline has great advantages. It allows development teams to deliver value to customers quickly and create truly agile development processes.
Learn more in our detailed guide to continuous delivery
Continuous Deployment (CDep) goes one step further than continuous delivery. All changes going through all stages of the production pipeline undergo automated tests, and if these tests pass, they are immediately deployed to production and exposed to customers.
Continuous deployment puts an end to release dates, and is a great way to speed up the customer feedback loop and reduce stress on the team. Developers can focus on building the software and see it running in production minutes after completion.
Continuous deployment can be difficult to implement. It requires seamless automation at all stages of the process, robust automated testing suites, and a culture of “continuous everything” that enables detection and rapid response to production issues.
Learn more in our detailed guide to continuous deployment
How Does CI/CD Relate to DevOps?
DevOps promotes better collaboration and communication between development (Dev) and operations (Ops) teams. It often requires changing various aspects of the development lifecycle, including job roles, tools, best practices, and automating the lifecycle.
DevOps typically involves the following:
- Adopting automation, programmable infrastructure deployment and maintenance, and iterative software development.
- Establishing cross functional teams while facilitating a culture change to build trust between these previously disparate teams.
- Aligning technologies to business requirements.
CI/CD supports the efforts of DevOps teams. It enables teams to implement automation across the development lifecycle and rapidly validate and deliver applications to end-users. Here is how it works:
- Continuous integration tools help initialize processes, ensuring developers can build, test, and validate code within a shared repository without manual work.
- Continuous delivery tools extend these automated steps to production testing and configuration for release management.
- Continuous deployment tools automatically invoke tests, handling configurations, provisioning, monitoring, and rollbacks.
What Are the Stages of a CI/CD Pipeline?
The CI/CD pipeline performs continuous integration, delivery, and deployment in four phases—source, build, test, and deploy.
Creating source code is the first phase in a CI/CD pipeline. During this phase, developers translate requirements into functional algorithms, features, and behaviors. Tools often vary, depending on the project, the project’s language, and other variables. As a result, there is no uniform source creation pipeline.
A source code creation pipeline may incorporate any of the following:
- A programming framework, such as Java, .NET, C#, or PHP.
- An integrated development environment (IDE) that supports the programming language chosen for the project.
- Code-checking tools, such as vulnerability scanners, basic error detection, and tools verifying adherence to coding standards.
- Code repositories and version control systems, such as Git.
The build phase involves pulling source code from a repository, establishing links to libraries, dependencies, and modules, and building these components into an executable (.exe) file. It typically requires tools that can generate execution logs, denote errors to correct and investigate, and notify developers once a build is completed.
Build tools vary according to the programming language. Some scenarios may require a specific build tool, while others can employ the same IDE for both source and build phases. A build phase may use additional tools to translate an executable file into a deployable or packaged execution environment, such as a virtual machine (VM) or a Docker container.
During the source code creation phase, the code undergoes static testing. The completed build enters the next CI/CD phase to undergo dynamic testing, including:
- Basic functional or unit testing—helps validate new features work as intended.
- Regression testing—helps ensure changes do not break previously working features.
In addition to functional and regression tests, the build undergoes tests that verify integration, performance, and user acceptance. If errors occur during the testing phase, the process loops these results back to developers for analysis and remediation. Since builds undergo many tests, developers employ automated testing to minimize human error and improve productivity.
After a build passes the testing phase, it becomes a candidate for deployment. There are two main ways to deploy the build, including:
- Continuous delivery—the build is sent to human staff for approval and then deployed. For example, new versions are automatically deployed to a test environment, but promotion to production is gated by a manual approval or merge request.
- Continuous deployment—the pipeline automatically deploys the build to testing, staging, and production environments, assuming it passes all relevant tests, with no manual approvals.
A typical deployment phase creates a deployment environment and moves the build to a deployment target, like a server. You can automate these steps with scripts or workflows in automation tools. Most deployments also integrate with error reporting and ticketing tools to detect unexpected errors post-deployment and alert developers.
Learn more in our detailed guide to the CI/CD pipeline
Here is a brief review of popular CI/CD tools.
Learn more about these and other tools in our detailed guide to CI/CD tools
Continuous Integration Tools
Popular CI tools include Codefresh, Bitbucket Pipelines, Jenkins, CircleCI, Bamboo, and GitLab CI.
Codefresh is a comprehensive GitOps continuous integration toolset designed for Kubernetes and modern applications. It is built from the ground up for flexibility and scalability around Argo Workflows and Argo Events. It takes the best of the open source toolset and provides essential enterprise features like a unified user interface, a single pane for cloud-wide management, security validated enterprise-grade runtime, end-to-end auditability, and cross-application single sign-on.
Bitbucket Pipelines is a CI tool that integrates directly into Bitbucket, a cloud-based source control system. It lets you manage pipelines as code and deploy your projects to production via CD tools. You can use Bitbucket pipelines to create pipeline definitions and kick off builds.
Jenkins is an open source automation tool that provides plugins to help develop, deploy, and deliver software. It is a server that lets developers distribute tasks across various machines and perform distributed tests and deployments. The Jenkins Pipeline offers several plugins to facilitate the implementation of a continuous integration (CI) pipeline.
CircleCI is a CI tool that supports various container systems, delivery mechanisms, and version control systems like Github. CircleCI can run complex pipelines with caching, resource classes, and Docker layer caching. You can run this tool in the cloud and on-premises.
Bamboo is an automation server for continuous integration that can automatically build, test, integrate, and document source code to prepare apps for deployment. It offers a simple user interface for CI/CD and various features, including automated merging and built-in deployment support.
GitLab CI is an open source CI tool. It lets you use the GitLab API to install and set up projects hosted on GitLab. GitLab CI can help you test and build projects and deploy your builds. It indicates areas that require improvement and lets you secure project data using confidential issues.
Continuous Delivery and Deployment Tools
Popular CD tools include Codefresh, Argo CD, GoCD, AWS CodePipeline, Azure Pipelines, and Spinnaker.
Codefresh is a modern GitOps software delivery solution powered by Argo with support for advanced deployments like canary, blue-green, and experimental releases. It provides comprehensive dashboards that offer visibility from code to cloud while integrating with your favorite tools. A centralized dashboard gives insight into deployments at scale while providing the security and support enterprises need.
Argo CD is a Kubernetes-native CD tool optimized for GitOps. It stores configuration in a Git repository and automatically applies it to Kubernetes clusters, making it easy to integrate with existing workflows. Argo CD can detect configuration drift, monitor application health, and roll back unwanted configuration changes. It also supports progressive delivery strategies like blue/green and canary deployment.
Learn more in our detailed guide to Argo CD
GoCD is an open source CD tool that helps automate the entire build-test-release process, including code check-in and all the way to deployment. It works with Git, Subversion, Mercurial, TFVC (TFS), and Perforce, and has an open plugin ecosystem. It is deployed on-premises.
AWS CodePipeline is a cloud-based CD service that helps model, visualize, and automate software release steps and continuous changes. Notable features include release process automation, establishing a consistent release process, and viewing pipeline history details.
Learn more in our detailed guide to CI/CD in AWS
Learn more in our detailed guide to CI/CD in Azure
Spinnaker is an open source CD platform for multi-cloud environments. It offers a pipeline management system and integrates with many cloud providers. Spinnaker provides a pipeline builder to automate releases, and lets you save and reuse existing pipelines as JSON files. It supports Kubernetes and integrates with tools like Prometheus, Datadog, and StackDriver.
CI/CD Best Practices
Here are a few best practices that can help you practice CI/CD more effectively.
Learn about these and other best practices in our detailed guide to CI/CD best practices
Build Only Once
Eliminate the practice of building the same source code multiple times. If you need to build, package, or bundle your software, you only need to perform this step once and promote binaries to the next stage of the pipeline.
Most successful CI implementations include the build process as the first step in the CI/CD cycle, making sure that software is packaged in a clean environment. This eliminates human error and reduces the chance of overlooked artifacts or incorrect artifacts included by mistake. Also, any artifacts generated must be versioned and uploaded to Git, to ensure that every time they are needed in the process, the same version of the build is available.
Prioritize Automation Efforts
Organizations moving to automated processes often struggle to identify which processes to automate first. For example, it is useful to automate the code compilation process from scratch. It is a good idea to run automated smoke tests every time developers commit new code. Unit tests are usually automated first to reduce developer workload.
In most cases, you will automate functional testing before UI testing. Unlike UI tests, which change more frequently, functional tests do not require frequent updates to automation scripts. Consider all possible dependencies, assess their impact, and prioritize automation as appropriate.
A commercial release is only possible if the software is release-ready and tested in a production-like environment. Therefore, it is best to add a step that deploys new versions to a realistic pre-production staging environment, or to the production environment itself alongside the current production version.
The following are release strategies that can help you deploy software to staging and production environments with low risk:
- Canary deployment—release the new version for some users, test their response, and if it works well, roll it out to a larger population. If the test fails, roll back and repeat.
- Blue/green deployment—run the current and new version of the software in two identical production environments. Initially, the current version is live and the new version is idle. Then traffic is switched over from the current version to the environment containing the new version. This lets you test the new version on real user traffic, and if something goes wrong, you can immediately roll back to the current stable version.
- A/B Testing—A/B testing is a method used to test the functionality of an application, such as changes to the user experience. Two or more versions of the application, with small differences between them, are served to production users in parallel. Teams observe how users interact with each version, and when one version is deemed successful, it is rolled out to all users.
Make the CI/CD Pipeline the Only Way to Deploy to Production
Investing in building a reliable, fast, and secure CI/CD pipeline gives you confidence in your build quality, but bypassing that process for any reason can hurt your efforts. Requests to circumvent the release process often occur because changes are minor or urgent—you should not give in to these requests.
Skipping automated tests runs the risk of production issues, but the problem does not end there. It is much more difficult to reproduce and debug problems, and trace them to specific build artifacts, because builds are not automatically deployed to test and production environments.
Even if at some point a team makes an exception and skips the CI/CD process, it is worth understanding the motive. Why was the request to skip the CI/CD pipeline made in the first place? Talk to key stakeholders and identify if the process seems slow or inflexible to them. You may need to make performance or process improvements to address those concerns.
By remaining responsive to stakeholder requirements, and communicating the benefits of the CI/CD pipeline, you can convince stakeholders and avoid disrupting the CI/CD process due to urgent requests.
Clean Up Environments with Every Release
To get the most out of your testing process, it’s worth cleaning up pre-production environments before each deployment.
If your environment has been running for a long time, it can be difficult to keep track of all configuration changes and updates applied—this is known as configuration drift. This means tests may not return the same results. Maintaining a static environment incurs maintenance costs, slows down testing, and delays the release process.
By using containers to host environments and run tests, you can easily start and destroy each newly deployed environment by scripting these steps using declarative configuration (for example, Kubernetes YAML files). Instantiating new containers before each deployment ensures consistency, and makes it easy to scale your environment to test multiple builds in parallel.
Taking CI/CD to the Next Level with Codefresh
Codefresh has made it our mission since 2014 to help teams accelerate their pace of innovation. Our platform, powered by Argo, combines the best of the open-source with an enterprise-grade runtime allowing you to fully tap the power of Argo Workflows, Events, CD, and Rollouts. It provides teams with a unified GitOps experience to build, test, deploy, and scale their applications.
Conquer DevOps with Codefresh
Realize your true DevOps potential with the premier GitOps solution powered by Argo.Check It Out