Automated Visual Regression Testing with Percy.io
Motivation
Have you ever been in a situation where you have updated some components in your site and as an unintended effect you altered the look and feel of some other parts of your site and it went unnoticed for some time? If your answer is yes or you want to avoid it, then you’ll get great benefits of implementing visual regression testing for your site.
Visual regression is a technique used to compare a baseline set of approved screenshots to a newly taken set of screenshots in order to find differences that could introduce regressions in the styling of your site.
Percy.io is a visual review tool that helps you in this process of comparing screenshots. The tool will allow you to review the differences and mark them as approved or error depending on whether you actually want this change or not.
Setting up our testing environment
For this blog post, we’ll use a Drupal site hosted on Pantheon and the Continuous Integration is set up using CircleCI. The continuous integration is set up in 4 stages: build, deploy, deploy-test, deploy-live-hold and deploy-live. The deploy-live-hold stage has two jobs in it: a hold job for manual approval before live deployment and visual-regression. Assuming test and live will have the same database, the visual regression testing will compare (after deployment) a set of screenshots between those different environments.
You need to start by creating a percy.io account and create a new project within that account. Once done, look for the PERCY_TOKEN inside the project settings page. You’ll need this value later.
There are several ways to implement Percy in your project, we’ll follow what’s probably the most basic way to do it. We only need the percy CLI tool which is written in nodejs and some graphical UI stuff that it’s probably already installed in your computer.
To install the CLI you need to install the npm package as a local package by running:
npm install -D @percy/agent
Then, you need to create the test script. Save it into a file inside your repo. An initial example could be the following:
/**
* @file
* Percy snapshots.
*/
/* eslint-env node */
‘use strict’;
const PercyScript = require(‘@percy/script’);
PercyScript.run(async (page, percySnapshot) => {
await page.goto(process.env.BASE_URL);
await page.waitFor(‘#block-particle-content’);
await percySnapshot(‘homepage’);
await page.goto(process.env.BASE_URL + ‘/blogs’);
await page.waitFor(‘#block-particle-content’);
await percySnapshot(‘blog-listing’);
await page.goto(process.env.BASE_URL + ‘/projects’);
await page.waitFor(‘#block-particle-content’);
await percySnapshot(‘projects-listing’);
await page.goto(process.env.BASE_URL + ‘/contact’);
await page.waitFor(‘#block-particle-content’);
await percySnapshot(‘contact’);
await page.goto(process.env.BASE_URL + ‘/blog/preparing-your-site-drupal-9’);
await page.waitFor(‘#disqus_thread’);
await percySnapshot(‘blog-item’);
await page.goto(process.env.BASE_URL + ‘/projects/hipatia’);
await page.waitFor(‘#block-particle-content’);
await percySnapshot(‘project-item’);
});
In this example I’m using BASE_URL to get the site base url to start running the tests. To run the tests, you need to execute the following commands:
export BASE_URL=”https://kporras07.com"
export PERCY_TOKEN=”token”
./node_modules/.bin/percy exec — node percy/snapshots.js
This will create the screenshots set in your percy account and will compare them to the last approved set of screenshots.
To automate this process, you could add it to your CI pipeline. If you do that, you need to first install some dependencies like this:
apt -y install nodejs gcc g++ make gconf-service libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libcairo2 libcups2 libdbus-1–3 libexpat1 libfontconfig1 libgcc1 libgconf-2–4 libgdk-pixbuf2.0–0 libglib2.0–0 libgtk-3–0 libnspr4 libpango-1.0–0 libpangocairo-1.0–0 libstdc++6 libx11–6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxss1 libxtst6 libappindicator1 libnss3 libasound2 libatk1.0–0 libc6 ca-certificates fonts-liberation lsb-release xdg-utils wget
The full section of CircleCI config file for this example looks like this:
visual-regression:
docker:
- image: kporras07/docker-drupal-nginx:php-7.4.x
working_directory: /var/www/kporras07
steps:
- attach_workspace:
at: /var/www
- run:
name: Run Snapshots
command: |
apt update -y
apt -y install nodejs gcc g++ make gconf-service libxext6 libxfixes3 libxi6 libxrandr2 libxrender1 libcairo2 libcups2 libdbus-1–3 libexpat1 libfontconfig1 libgcc1 libgconf-2–4 libgdk-pixbuf2.0–0 libglib2.0–0 libgtk-3–0 libnspr4 libpango-1.0–0 libpangocairo-1.0–0 libstdc++6 libx11–6 libx11-xcb1 libxcb1 libxcomposite1 libxcursor1 libxdamage1 libxss1 libxtst6 libappindicator1 libnss3 libasound2 libatk1.0–0 libc6 ca-certificates fonts-liberation lsb-release xdg-utils wget
export BASE_URL=”https://test-kporras07.pantheonsite.io"
./node_modules/.bin/percy exec — node percy/snapshots.js
When a percy test is run, it should appear in your Percy UI and it will show you the build url. When you open that, it will look like this:
Conclusion
Visual regression testing is an excellent choice to avoid visual regression errors and Percy.io is a good and easy to setup tool. Its free plan allows up to 5000 screenshots/month; so, it’s probably a good choice if your demand is not so high. It’s definitely a good tool to check out and you may want to give it a try.
