Schedule a FREE onboarding and start making pipelines fast.

Selenium Deployment Verification Testing of Docker’s Example Voting App

In this blog post, I will be focusing on running Deployment Verification Tests against Docker’s Example Voting App using Selenium and PyTest against the Kubernetes Helm Release we worked on in Part 1.

New to Codefresh? Get started with Codefresh by signing up for an account today!

This blog is Part 2 of a series The Perfect Pipeline.

Adding 4 New Build Steps:

  • GetKubernetesServicesEndpoints: We’re gathering the Endpoint IPs for vote and result service’s from Kubernetes
  • BuildingTestDockerImage: Building a Test Docker image with Python, PyTest and Selenium..
    PostDeploymentVerificationTests: Running a composition with Selenium Hub, Chrome Browser, Firefox Browser and Test image. This composition will utilize the open source Selenium Docker images to run our PyTest Deployment Verification Tests.
  • ArchiveDVTs: We’re generating HTML reports in the prior step and in this step archiving them in S3 and annotating our Test Docker image’s metadata with the URLs to the reports.

The diagram below represents the workflow discussed in this blog post.

Update to Existing YAML

Some updates have occurred to names and titles of the build steps but below are the critical updates.

Edit value of RELEASE_NAME for the HelmUpgrade build step to ${{HELM_RELEASE_NAME}}.
The Helm Release Name was templated for later use in new build steps.

Please be aware that WordPress may have modified symbols used the scripting below.

All examples below are available in this Github repository.

https://github.com/codefresh-io/example-voting-app

Create PyTest + Selenium python script

Create /tests/selenium/test_app.py example below..

There are simple tests for both the vote and result service’s webpage.

Using PyTest + Selenium Hub and allowing for multiple browsers.

For vote service we’re checking:

  • the value of the web page’s title is Cats vs Dogs!
  • the existence of the choice element
  • the existence of button a
  • the existence of button b

For result service we’re checking:

  • the value of the web page’s title is Cats vs Dogs -- Result
  • the existence of the result element

We’re also doing a simple button click() test on the vote web page and confirming the result web page tallied that vote.

  • execute click() of button a on vote web page
  • confirm result element of the result web page does not equal No votes yet as this would signify we didn’t tally the click() to button a on the vote web page.

Create Dockerfile for Test Docker image

Now we’ll create a new Dockerfile in the root folder of the repository

We’re skipping copying in our tests in this Dockerfile and instead will use tests mounted from the GIT repository in our composition candidate or our composition build step.

Now that we have a Docker image ready for our Codefresh Pipeline we need to add a new build step to our Pipeline’s codefresh-matrix-pipeline.yml to build the Docker image.

Add Docker build Build Step for Test image

Add Get Kubernetes Service IP Build Step

Add GetKubernetesServicesEndpoints build step to get Endpoint IPs for vote and result service.

To support the changes made in the matrix YAML file we need to add the following Environment Variables.

Add Helm Release Name and Kubernetes Services as Environment variables on the matrix pipeline.

  • HELM_RELEASE_NAME = example-voting-app <- YAML supports any name you’d like to use.
  • services = vote result <- space delimited services we would like to return Kubernetes External IPs for.

You will need to add additional browsers to the composition build step PostDeploymentVerificationTests if you’d like to support additional browsers.

Add Selenium Deployment Verification Build Step

Now we’ll add the composition build step which will stand up Selenium Hub, Chrome and Firefox browsers as supporting infrastructure and join in our Test Docker image as a composition candidate.

We’ve setup the composition candidate with a volume mount of our Github clone and passed in our Kubernetes Service’s IPs and browsers to test against.

When the tests have finished we will annotate the Test Docker image’s metadata with pass/fail.

Add supported browsers as Environment variables on the matrix pipeline.

  • browsers = chrome firefox <- space delimited browsers you’d like to execute Selenium DVTS against.

You will need to add additional browsers to the composition build step PostDeploymentVerificationTests if you’d like to support additional browsers.

Add a S3 Bucket and Upload PyTest HTML Reports to S3 bucket

We generated HTML reports for each browser and will upload these to S3 bucket for the build and add the URLs to the Test Docker image’s metadata.

NOTICE

LINE 123: I’ve added an argument to make the uploads public --acl public-read

You may want to remove this argument if you use this build step in a private setting.

Prerequisites:

S3 bucket:
https://docs.aws.amazon.com/AmazonS3/latest/gsg/CreatingABucket.html

AWS Access Key and Secret Access Key:
https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys

Add YAML below to codefresh-matrix-pipeline.yml

To support the changes made in the matrix YAML file we need to add the following Environment Variables.

Add AWS Region and Credentials to AWS_CLI Shared Secret

Account Settings -> Shared Configuration -> ADD CONFIGURATION CONTEXT -> Shared Secret

AWS_DEFAULT_REGION = aws-region-s3-bucket <- S3 bucket’s AWS Region.
AWS_ACCESS_KEY_ID = aws-access-key <- Access Key with read/write access to S3 bucket.
AWS_SECRET_ACCESS_KEY = aws-secret-access-key <- Secret Access Key for Access Key above.

Add S3 Bucket Name Global Variable

Codefresh Repository -> General (tab) -> Global Variables

S3_BUCKETNAME = example-voting-app <- named after my repository

We’ll be using this in future blog posts across the different pipelines for the Codefresh Repository.

Resources:

https://github.com/codefresh-io/example-voting-app/tree/add-selenium-tests#part-2-add-selenium-deployment-verification-testing-of-deployment

http://pytest-selenium.readthedocs.io/en/latest/index.html

https://github.com/SeleniumHQ/docker-selenium

https://github.com/SeleniumHQ/selenium/tree/master/py/test/selenium/webdriver/common

Planning for next blog to focus on Helm Chart Management using Codefresh’s embedded Chart Museum and use of Helm Repository in CI/CD workflow.

Codefresh Helm Repository

Hope, you find this tutorial useful. I look forward to your comments and any questions you have.

New to Codefresh? create a free Codefresh account and start building, testing and deploying Docker images faster than ever.

About Dustin Van Buskirk

A Senior Solution Architect at Codefresh

Reader Interactions

Enjoy this article? Don't forget to share.

Comments

Your email address will not be published. Required fields are marked *

Follow me on Twitter