A few years ago I was excited about the idea of visual regression testing


A few years ago I was excited about the idea of visual regression testing. (Visual regression tests take a picture of a component or page and then diff with the latest code so you're notified when rendering changes).

There are a number of tools in this space. I've used Chromatic, Percy, and Happo.

But I found visual tests to be tricky to do well.

Three examples:
1. Small pixel differences were sometimes reported when no code actually changed.

2. In fast moving apps, UI changes occurred so often that it was annoying to constantly diff pictures and click approve.

3. For testing apps, I needed either ubiquitous mock data, or a stable DB dataset to assure the visual tests always "see" the same thing. This is a lot of work to setup and maintain.

Bottom line: Visual regression testing can be useful for stable, reusable components. But I avoid using for testing applications.

View original on X