How to Measure QA Success
The benefits of quality assurance testing in software are widely acceptedRead
Nearly everyone agrees that QA test workflows are imperfect when they’re written. That’s because it’s impossible to foresee every use case for the workflow and to ensure that it will continue to work after dozens of updates to that section of the codebase.
This is especially true in the world of email testing, where emails are constantly getting updated, rebranded, refreshed, and analysed.
So, it’s no surprise that there’s a need for someone to be responsible for the ownership of the QA test workflow.
Before we jump into who is (or isn’t) responsible for improving the QA test workflow, let’s start with a basic question: what is a QA test workflow?
A workflow is essentially the checklist of things that need to be tested for a piece of software to pass the QA test. That may sound simple, but it’s pretty difficult to get it right.
Let’s take the webpage you’re on right now as an example. To do a thorough QA test, you’d need to check every link (let’s say there’s 25), in every popular browser (5), on every popular device (10), in multiple languages. Now, for this single page alone, that is thousands of discrete tasks.
Now, while that example is a bit extreme — since most developers will make sure that you don’t need to test every link in each browser, since if it works in one place it should work in every place — for the most commonplace of QA tests, there will be dozens, if not hundreds, of tasks to check off of a list.
That’s where the workflow comes in: it makes sure that you don’t forget any of the tasks you need to complete a QA test, it helps make sure there’s documentation on what’s getting tested, and it ensures that regardless of who is doing the testing, the same tests are getting done.
In an email, these tasks typically include undergoing a grammatical and typo check, previewing the visual results of the email, checking all links and alt-tags, checking that analytics are registering, checking that all names and personal information are being populated in the right place, and verifying that edge cases are being accounted for.
It sounds straightforward, but managing a workflow becomes complicated even for the simplest pieces of software.
So, with all of that work going into actually using a workflow, the question immediately arises: who is responsible for setting it up? And who is responsible for improving the workflow once it’s set up and in use?
Now, a smart organization will automate as much as possible of the repetitive and time-wasting QA tests that they need to perform. That means there’s initially a high degree of technical sophistication needed to setup a proper workflow. Naturally, this would seem to indicate that developers are responsible for maintaining and improving QA testing workflows.
At the same time, once the software has shipped, the original developer may not touch the code again, while the QA testing team is responsible for making sure it continues to work following other changes to the code base. This would seem to indicate that the QA team is responsible for the QA testing workflow. The truth is that there are a bunch of different options for maintaining and improving QA testing workflows, which is what makes it so crucial to have a clear owner who is responsible for the maintenance of a workflow.
Without that clear ownership, you’ll end up with a workflow with no owner, that no longer tests well, and leaves gaps in your QA processes. That’s a situation that no software company wants to be found in.
In our view, there are three main options for who will manage your QA test workflows.
These apply to the vast majority of software companies, and while there are edge cases when one is better than the other (or when other options are available), it’s our view that these make the most sense for the largest number of companies.
The first option is to go to a third-party that handles all of your testing and test-management needs. This is something that plenty of companies try at some point in time, but which, for reasons we’ll now explain, never stick with in the long-term.
An option that a lot of companies use when they’re just getting started doing QA testing is having the QA team contact the developers to update the testing workflows whenever an update is necessary. It’s essentially having the developers make any changes at the request of the QA team.
The final option is the QA team itself taking ownership of the entire workflow development process. From the very beginning of defining the tests, to the end stage where the workflow is being maintained and improved as the need arises.
Instead of deciding on one owner, we believe in going the pragmatic route: have two owners for different parts. Now, before everyone gets their hackles raised and starts questioning why we would have two owners — “Doesn’t that defy the idea of ownership and responsibility?” — let’s be clear: at the end of the day, QA testing is owned by the QA Team.
But. There are two parts to owning QA test workflows that need to be considered: the initial setup and ongoing maintenance.
In our view, the initial setup belongs to the engineers, since they’re already deeply familiar with the specifications of the software and are in the code-base.
Once the initial workflow is encoded, the baton of ownership is then handed off to the QA team, who will be responsible for improving and maintaining the workflow. This is because they’re the end-users and know exactly what needs to be changed as time goes on.
This transfer of ownership means the people with the right skills are working on the workflow at the right stage, maximizing advantages, and minimizing disadvantages.