Codefresh adds built-in reporting for test results and other custom CI/CD reports

Codefresh adds built-in reporting for test results and other custom CI/CD reports

4 min read

One of the pillars of Continuous Integration is the active use of tests as part of the integration process. Each feature is considered “done” only when it passes all required tests on its own as well as part of a release.

Codefresh has always supported the running of unit and integration tests as part of any pipeline. In fact, since Codefresh pipelines use Docker images as build steps, running unit tests was always possible for any kind of programming tool and framework that was offered in the form of a Docker image. This means that no matter your programming language, Codefresh can easily run your tests regardless of the test framework you choose.

Running unit tests is, however, only half of the story. The other part is providing actionable test reports that are interesting to all project stakeholders.

Today Codefresh is offering an integrated pipeline step that produces a rich set of graphical test reports including historical data.

Example test report
Example test report

The graphical test report is based on the open-source Allure project. The supported languages/frameworks include the following:

  • Java/JUnit/TestNG/Cucumber
  • Python/Pytest
  • Javascript/Jasmine/Mocha
  • Ruby/Rspec
  • Groovy/Spock
  • .NET/Nunit/Mstest
  • Scala/Scalatest
  • PHP/PhpUnit

Connecting the storage provider that holds the reports

First, you need to connect your cloud storage provider to Codefresh. Currently, both Google Cloud and Amazon web services are supported.

In both cases, you create a storage bucket that will be responsible for storing the reports for each build. Then you can easily connect the provider in the Codefresh Integrations screen.

Cloud storage setup
Cloud storage setup

Once that is done, the only thing Codefresh needs to know is the name of the bucket that will store the reports.

Collecting the test results

After cloud storage is set up, you are ready to run unit/integration tests and have Codefresh collect the results. See the official Allure documentation on how to gather the information needed by the test report.

An example of Javascript/NPM would be:

running_tests:
   image: node
   commands:
    - npm test
    - cp -r -f ./allure-results /tmp/

What Codefresh offers is a premade build step that takes the results and uploads the reports to Codefresh infrastructure. In your codefresh.yml file, you only need the following:

unit_test_reporting_step:
   title: Generate test reporting
   image: codefresh/cf-docker-test-reporting
   working_directory: /tmp
   environment:
     - BUCKET_NAME=my-bucket-name
     - CF_STORAGE_INTEGRATION=google

The working directory is any folder of your choice that contains the Allure results in a subfolder called allure-results. In the example above, we assume that the tests saved their results in /tmp/allure-results. We also specify that we will use Google integration and upload the reports to the bucket named my-bucket-name.

That’s it. You can visit your test reports for any build by clicking on the respective button found in the GUI. You can also easily share the link with your colleagues.

Viewing test reports
Viewing test reports

Uploading custom reports

As a further enhancement to this feature, we are also allowing our customers to upload any kind of HTML report to the Codefresh infrastructure (such as security scanning or code quality results). This way Codefresh can serve as the central place for all kinds of reports, both for Continuous Integration and Continuous Delivery.

Even though Allure is a great framework and supports a lot of programming tools and languages, we wanted to offer a flexible solution where customers can upload reports from any other tool or utility (as long as they produce HTML report).

The way to do this is just by adding two more parameters in the test reporting step as seen below:

unit_test_reporting_step:
   title: Upload Mocha test reports
   image: codefresh/cf-docker-test-reporting
   working_directory: /codefresh/volume/demochat/
   environment:
     - REPORT_DIR=mochawesome-report
     - REPORT_INDEX_FILE=mochawesome.html
     - BUCKET_NAME=my-bucket-name
     - CF_STORAGE_INTEGRATION=google

The two additional parameters define a custom folder in your project that contains the reports as well as which file will serve as the index file.

Here is an example for Mocha reports:

Mocha Tests
Mocha Tests

And here is another example of Gradle reports

Gradle Reports in Codefresh
Gradle Reports in Codefresh

In summary, every kind of report from any tool can now become an integral part of a Codefresh pipeline.

For more details see the official documentation.

New to Codefresh? Create Your Free Account Today!

How useful was this post?

Click on a star to rate it!

Average rating 0 / 5. Vote count: 0

No votes so far! Be the first to rate this post.

4 thoughts on “Codefresh adds built-in reporting for test results and other custom CI/CD reports

  1. Is there any reason why I can’t see the trends in the allure report. And also is there any way to Integrate reports from multiple pipelines to single report in the pipeline. Right now to get combined reports I am testing everything and generating the report in the same pipeline. Thanks.

    1. Hello. Historical trends should be supported out of the box. It is better to open a support ticket with us and include the URL of the build that has the issue.

  2. Is there a way to download the results? I’m uploading result files using AWS integration and upload the reports to the bucket. Although, I can view report using the link attached in the pipeline build, I’m not how long these results would be stored there. So, is there a way I can archive results file and download it when build is over?

    1. Hello

      The results should stay there forever. They are in an S3 bucket that you (or your admin) control. If you want to archive you just download them from S3
      like any other file. Ask you admin how your object storage is backed up.

Leave a Reply

Your email address will not be published. Required fields are marked *

Comment

Build your GitOps skills and credibility today with a GitOps Certification.

Get GitOps Certified

Ready to Get Started?
  • safer deployments
  • More frequent deployments
  • resilient deployments