One of my colleagues from the Screenster team once told me that regression testing is the single best reason why QA automation exists. According to the same person, doing regression testing the right way is a major challenge for most web development projects.
So what is so challenging about it? And more importantly, why taking regression testing to the cloud is a good idea?
Why do web teams have problems automating regression testing?
Two words, “UI testing”. It’s common knowledge that the ROI of automated UI regression testing is low. I’m sure you’re familiar with the culprits of this problem:
- Even with skilled professionals onboard, most teams struggle with the maintenance of coded UI regression tests.
- QA automation frameworks often lack efficient collaboration tools. In fact, few solutions will even feature shared integrated portals displaying test statuses.
- Existing testing environments require manual setup and maintenance. Specifically, frameworks often make you code the fundamentals of the framework itself before you can actually start writing your tests.
Take another look at this list, and you’ll see that most of the problems stem from the inherent shortcomings of the tools, not the process. So maybe it’s time we moved on to something different?
Taking regression testing to the cloud: the case of AjaxSwing
What do most teams that struggle with UI testing automation have in common? They code tests using Selenium :-). In fact, my team had the exact same problem in the past. Trying to develop automated Selenium tests for our product AjaxSwing was a painful experience, so we started searching for alternatives.
At some point, we found several cloud-based record-playback tools that seemed to “get it right”. While each platform had it’s shortcomings, they all shared the same major advantages. Namely, these cloud-based solutions ditched hand-coded UI tests. Besides, most of these tools had decent collaboration features available out of the box.
Visual tools for UI regression testing on the cloud
Long story short, none of the cloud-based solutions that existed at that point worked for us because of the small drawbacks each of them had. On the bright side, we got a chance to see what worked for real-life applications and what didn’t. So we picked the features we considered best and used them to build our own platform Screenster.
With Screenster, our team strived to cover the functionality we consider must-have for any tool that deals with regression testing of web UIs. Namely, these are the features that you should expect from any cloud-based regression-testing platform:
1. Tests should cover the whole UI state, not just target separate elements
Targeting separate elements of the frequently changing UI with hundreds of coded tests is the recipe for maintenance hell. Thing is, no matter how good you are at writing UI tests, these tests will never cover all the possible issues brought by CSS updates. So maybe it’d be wiser to leave this grunt work to the machine?
Instead of making you write hundreds of tests for dozens of UI components, platforms like Screenster scan the complete UI state of the page you’re on. And they can diff these ‘scans’ during regression testing to guarantee that no issue goes unnoticed.
2. Collaboration functionality is paramount
Most projects involve teams, not just individuals. Given this fact, it’s weird that so many enterprise solutions neglect collaboration features. By definition, cloud-based solutions can store test suits on a shared portal. The exact set of collaboration tools available out of the box differs depending on a particular solution, so the best way to see which one suits you best is to play around with it.
3. No setup is the best setup
Decent cloud-based solutions do everything to minimize setup and installation pain. Ghost Inspector substitutes installation packages with a Chrome plugin paired with a web app. Besides, there are tools that offer a somewhat more sophisticated solutions by integrating a new menu into Chrome Developer Tools.
The complete functionality of Screenster is available via our web app — simple as that. In case you do need to run regression testing on premise, there is an option to set up a shared server. But even in that case, you’ll appreciate the simplicity of the process.
4. UI regression testing should be accessible for non-technical users
To me, the idea of hand-coding UI tests feels like hammering nails with a microscope. And having a professional programmer do this is akin to hiring a PhD in Biology to do the hammering. Most team leads will agree that it’s best to let programmers write features (and unit tests for the said features). So why don’t we all start doing just that?
With a decent cloud-based platform, manual QAs and non-technical folks can run 99% of your UI regression tests. When working on Screenster, we’ve put a particular focus on the smooth learning curve and user friendliness. Screenster doesn’t require programming knowledge, and it handles things like timeouts, locators and dynamic areas automatically.
5. Record-playback coupled with smart verification
I guess you’d pretty much expect your regression testing tool to do a lot more than basic screenshot comparison or basic record-playback. Differences verification is an obvious must-have, so is the ability to analyze and verify the DOM structure and the CSS of the on-page elements.
Bottom line: to code or not to code?
I have to admit it, cloud-based regression testing tools like Screenster might make some QAs skeptical. After all, what’s the point in learning how to code if you can do just fine without code-based UI tests? But I’m sure that people who are able to see the bigger picture will appreciate the efficiency that cloud-based solutions offer.
Our team loves to code, but we want to code more features and less tests. The experience of using code-based tools for our product proved us that sometimes you have to look for a simpler solution. Reducing maintenance pain and allowing you to automate manual testing, Screenster offers 10x the productivity of a code-based UI testing solution.
So can UI testing teams ditch complex frameworks that require developers to write and maintain tests? The best way to see if it works for you project is to try it yourself.