How to run Unit tests in Codefresh pipelines
With Codefresh, you can easily run unit tests for every commit or pull request.
For the purposes of this guide, “unit tests” are the tests that use only the source code of the application and nothing else. If you are interested in running tests with external services (such as databases) then see the page about integration tests.
Different companies have different types of unit tests and in several cases the type of programming language also affects when/what tests are run. Codefresh supports all test frameworks (including mocking frameworks) for all popular programming languages.
In this page we will see 4 ways of running unit tests in Codefresh:
- Running unit tests in a dockerfile (recommended only for smoke tests)
- Running unit tests with an external image (best for traditional/simple applications)
- Running unit tests in the application image (not recommended, but very popular)
- Running unit tests using a special testing image (the recommended solution for complex applications)
For an example application of the second and third ways see the unit test example page
Running Unit tests as part of a Docker build
A handy way to quickly test a Docker image, is by placing one or more smoke tests in the Dockerfile itself. The unit tests then execute when the image is built, and if they fail the image is not even created. Here is an example:
This kind of unit tests are transparent to Codefresh. They will just execute in a build step in the same manner as if you would build the image on your workstation.
A big disadvantage of this unit testing method, is that getting reports out of the docker image is not a straightforward process. On the other hand they are very easy to integrate in any workflow. The Codefresh pipeline simply checks out the code and builds a Dockerfile.
This technique is best used for a very small subset of unit tests that check the overall well being of a Docker image. The bulk of the tests should be executed outside the Docker build process as we will see in the next sections.
Running Unit tests using an external Docker image
The recommended way to run unit tests in Codefresh pipelines is to select a Docker image that has all the test tools that you need and define an explicit testing step in your pipeline (usually a freestyle step).
Here is an example where a JDK/Maven image is used to run unit tests:
The big advantage of this approach is that you can easily replicate your test environment in the Codefresh pipeline by selecting the appropriate image for your tests. You also get a clear overview on the test results. If they fail, the pipeline will automatically stop. You can change this behavior with the fail_fast property.
Notice that even if the example above, creates eventually a Docker image, you can still use this way of running unit tests for traditional applications that are not dockerized yet (e.g. VM-based applications).
Running Unit tests with the Application Docker image
In several cases (especially with dynamic languages) the Docker image that holds the application can also be re-used for unit tests. This is a very common technique for Node, Python and Ruby applications. In this case you can use the context feature of Codefresh to run a unit test step in the image that was created in a previous step:
Notice that here we use a Codefresh variable as the value of the
image property in the last step. This will make the unit test execute in the same docker container that was created in the second step of the pipeline.
This technique is certainly useful, but can be easily abused if you end up shipping testing tools in your production image (which is not recommended). If you find your production images filled with test tools and libraries it is better to use the technique in the next section which uses a different image for tests.
Running Unit tests with a dynamic Docker image
The ultimate way of running unit tests in Codefresh is by creating a specific image dedicated to unit tests. If Dockerhub doesn’t already contain an image that suits you, you should instead create your own.
This means that your application has two Dockerfiles. The main one that holds the application as a deployment artifact and a separate one that holds all the unit test libraries and tools that you need. Here is an example:
Notice that here we create two Docker images.
- The first docker image is created from
- Unit tests run in the context of that image (
- The production application uses another Dockerfile
This is one of the best ways to run unit tests (as well as integration tests) as it allows you to fine tune the test environment while still shipping to production only what is needed.
In the example above we used two different dockerfiles, but you could also use a single dockerfile with multi-stage builds and use the
target directive to stop the image build process at a previous layer (that has all the testing tools).
Creating Test reports
All the ways mentioned above for running unit tests (apart of the first one), can also be used for test reporting.
Read all about test results and graphs in the test reports page.