How it works today: one box many scripts
Regular CI is triggered by a commit that executes the entire lifecycle in single box.
So, you need to make sure that in this box (be it a VM or some container based box) you install all dependencies and tools that are required for compiling, testing, and packaging.
The situation is more complex when you need some End To End (UI or API based testing) so you need to make sure that additional components will be available.
When your application is micro-service based, for E2E / Integration tests you need to spin-up a subset of this application that may require additional resources like CPU, memory, and complex configurations to make it run.
And what about CD? Triggering deployment will require more time, more tools (aws cli, kuberenets cli, etc), authentication, and additional scripts.
So, we are pushing tons of unrelated tools, scripts, and data in one box.
And what about parallelization? Right, it will all run in the same box.
We took different approach in Codefresh
Our concept is to enable the user to run different steps in separates boxes, such as:
Compilation + Unit Test
Compilation will run in a build box that will include compiler, uni tests frameworks, and lifecycle tools (such as MVN, etc).
Integration tests will include multiple boxes
-One (or several) where you application is running.
-One where your integration tests will run.
CD:
– Same thing: separate box that will include data that is ONLY relevant for push. We have a specific steps for ECS, SWARM, and K8.
In addition we have a dedicated step for optimized Docker build and push so you don’t have a deal with the Docker CLI.
Shared context between steps:
When the pipeline contains multiple steps, we maintain shared context that enables shared data. So, once the result of step is running in one box, it can be accessible by the other.
All this provides several game changing capabilities for our users :
- Ability to create complex CI and, especially, CD scenarios by reusing Docker Images.
- The pipeline totally isolated from virtual machine- We don’t keep any pipeline related resource on the host, so we can run multiple pipelines on same host .
- Debug easily – Pipeline is fully containerized so it will run same on different host including local machine.
- Running integration test on every commit – ability to spin up full application in context of specific pipeline make it super easy to run integration test on every commit.
And finally, smooth integration with deployment platforms by providing dedicated steps.
We believe that this approach resonates much more with building CI/CD to modern micro serviced and containerization applications.