Schedule a FREE onboarding and start making pipelines fast.

Codefresh adds built-in reporting for test results and other custom CI/CD reports

Codefresh News | November 28, 2018

One of the pillars of Continuous Integration is the active use of tests as part of the integration process. Each feature is considered “done” only when it passes all required tests on its own as well as part of a release.

Codefresh has always supported the running of unit and integration tests as part of any pipeline. In fact, since Codefresh pipelines use Docker images as build steps, running unit tests was always possible for any kind of programming tool and framework that was offered in the form of a Docker image. This means that no matter your programming language, Codefresh can easily run your tests regardless of the test framework you choose.

Running unit tests is, however, only half of the story. The other part is providing actionable test reports that are interesting to all project stakeholders.

Today Codefresh is offering an integrated pipeline step that produces a rich set of graphical test reports including historical data.

Example test report
Example test report

The graphical test report is based on the open-source Allure project. The supported languages/frameworks include the following:

  • Java/JUnit/TestNG/Cucumber
  • Python/Pytest
  • Javascript/Jasmine/Mocha
  • Ruby/Rspec
  • Groovy/Spock
  • .NET/Nunit/Mstest
  • Scala/Scalatest
  • PHP/PhpUnit

Connecting the storage provider that holds the reports

First, you need to connect your cloud storage provider to Codefresh. Currently, both Google Cloud and Amazon web services are supported.

In both cases, you create a storage bucket that will be responsible for storing the reports for each build. Then you can easily connect the provider in the Codefresh Integrations screen.

Cloud storage setup
Cloud storage setup

Once that is done, the only thing Codefresh needs to know is the name of the bucket that will store the reports.

Collecting the test results

After cloud storage is set up, you are ready to run unit/integration tests and have Codefresh collect the results. See the official Allure documentation on how to gather the information needed by the test report.

An example of Javascript/NPM would be:

What Codefresh offers is a premade build step that takes the results and uploads the reports to Codefresh infrastructure. In your codefresh.yml file, you only need the following:

The working directory is any folder of your choice that contains the Allure results in a subfolder called allure-results. In the example above, we assume that the tests saved their results in /tmp/allure-results. We also specify that we will use Google integration and upload the reports to the bucket named my-bucket-name.

That’s it. You can visit your test reports for any build by clicking on the respective button found in the GUI. You can also easily share the link with your colleagues.

Viewing test reports
Viewing test reports

Uploading custom reports

As a further enhancement to this feature, we are also allowing our customers to upload any kind of HTML report to the Codefresh infrastructure (such as security scanning or code quality results). This way Codefresh can serve as the central place for all kinds of reports, both for Continuous Integration and Continuous Delivery.

Even though Allure is a great framework and supports a lot of programming tools and languages, we wanted to offer a flexible solution where customers can upload reports from any other tool or utility (as long as they produce HTML report).

The way to do this is just by adding two more parameters in the test reporting step as seen below:

The two additional parameters define a custom folder in your project that contains the reports as well as which file will serve as the index file.

Here is an example for Mocha reports:

Mocha Tests
Mocha Tests

And here is another example of Gradle reports

Gradle Reports in Codefresh
Gradle Reports in Codefresh

In summary, every kind of report from any tool can now become an integral part of a Codefresh pipeline.

For more details see the official documentation.

New to Codefresh? Create Your Free Account Today!

Kostis Kapelonis

About Kostis Kapelonis

Kostis is a software engineer/technical-writer dual class character. He lives and breathes automation, good testing practices and stress-free deployments.

Reader Interactions

Enjoy this article? Don't forget to share.

Comments

Your email address will not be published. Required fields are marked *

Follow me on Twitter