One of the pillars of Continuous Integration is the active use of tests as part of the integration process. Each feature is considered “done” only when it passes all required tests on its own as well as part of a release.
Codefresh has always supported the running of unit and integration tests as part of any pipeline. In fact, since Codefresh pipelines use Docker images as build steps, running unit tests was always possible for any kind of programming tool and framework that was offered in the form of a Docker image. This means that no matter your programming language, Codefresh can easily run your tests regardless of the test framework you choose.
Running unit tests is, however, only half of the story. The other part is providing actionable test reports that are interesting to all project stakeholders.
Today Codefresh is offering an integrated pipeline step that produces a rich set of graphical test reports including historical data.
The graphical test report is based on the open-source Allure project. The supported languages/frameworks include the following:
- Java/JUnit/TestNG/Cucumber
- Python/Pytest
- Javascript/Jasmine/Mocha
- Ruby/Rspec
- Groovy/Spock
- .NET/Nunit/Mstest
- Scala/Scalatest
- PHP/PhpUnit
Connecting the storage provider that holds the reports
First, you need to connect your cloud storage provider to Codefresh. Currently, both Google Cloud and Amazon web services are supported.
In both cases, you create a storage bucket that will be responsible for storing the reports for each build. Then you can easily connect the provider in the Codefresh Integrations screen.
Once that is done, the only thing Codefresh needs to know is the name of the bucket that will store the reports.
Collecting the test results
After cloud storage is set up, you are ready to run unit/integration tests and have Codefresh collect the results. See the official Allure documentation on how to gather the information needed by the test report.
An example of Javascript/NPM would be:
running_tests:
image: node
commands:
- npm test
- cp -r -f ./allure-results /tmp/
What Codefresh offers is a premade build step that takes the results and uploads the reports to Codefresh infrastructure. In your codefresh.yml file, you only need the following:
unit_test_reporting_step:
title: Generate test reporting
image: codefresh/cf-docker-test-reporting
working_directory: /tmp
environment:
- BUCKET_NAME=my-bucket-name
- CF_STORAGE_INTEGRATION=google
The working directory is any folder of your choice that contains the Allure results in a subfolder called allure-results. In the example above, we assume that the tests saved their results in /tmp/allure-results. We also specify that we will use Google integration and upload the reports to the bucket named my-bucket-name.
That’s it. You can visit your test reports for any build by clicking on the respective button found in the GUI. You can also easily share the link with your colleagues.
Uploading custom reports
As a further enhancement to this feature, we are also allowing our customers to upload any kind of HTML report to the Codefresh infrastructure (such as security scanning or code quality results). This way Codefresh can serve as the central place for all kinds of reports, both for Continuous Integration and Continuous Delivery.
Even though Allure is a great framework and supports a lot of programming tools and languages, we wanted to offer a flexible solution where customers can upload reports from any other tool or utility (as long as they produce HTML report).
The way to do this is just by adding two more parameters in the test reporting step as seen below:
unit_test_reporting_step:
title: Upload Mocha test reports
image: codefresh/cf-docker-test-reporting
working_directory: /codefresh/volume/demochat/
environment:
- REPORT_DIR=mochawesome-report
- REPORT_INDEX_FILE=mochawesome.html
- BUCKET_NAME=my-bucket-name
- CF_STORAGE_INTEGRATION=google
The two additional parameters define a custom folder in your project that contains the reports as well as which file will serve as the index file.
Here is an example for Mocha reports:
And here is another example of Gradle reports
In summary, every kind of report from any tool can now become an integral part of a Codefresh pipeline.
For more details see the official documentation.
New to Codefresh? Create Your Free Account Today!