It’s a question that has beguiled an endless list of QA teams: should I automate this test? It’s not that there’s a preference in one direction or the other, it’s just that with an engineering mindset comes the desire to do the more efficient, practical thing. Sometimes that means automating a QA test, while other times it means continuing to manually perform the QA test yourself.
We’re not going to pretend that we have the answer for every single software team out there. In reality, every team is different and the priority that your team puts on particular tests can be wildly different from QA team to QA team. Instead, we believe there are a few things you can look at to figure out if your QA tests should be automated, or if you should stick to the manual tests you’ve been performing this entire time.
To start though, we need to understand the foundation of the question, which means understanding what QA automation is and why it matters in the first place.
Why Automation Matters
We hear all the time that automation matters, but have you ever stopped to consider why? Aside from just thinking that you’re more efficient, there are three big reasons why automation matters to a QA engineer or team manager.
Be Smart With Your Time
The most obvious reason you should automate your QA tests is that it will save you time. Don’t want to go through the same regression test in your settings dialogue? There’s an app for that! Or, there will be, once you write it. Automation is about saving time and resources so that you can devote them to things that require not just time, but also your brain power
Not just the time you save from not having to do the same manual test over and over again — that time-saving is obvious. No, you also save time from having a higher-quality result to your QA tests, having standardized pass/fail readings, and being able to focus on other work without getting distracted by another manual QA test.
Avoid Mistakes
It’s something we’ve all thought at times: QA testing can be incredibly monotonous. Not all of the time, but it’s common enough that we’ve all faced the same thought. That monotony can also be called momentary boredom. Unfortunately, we all know that boredom inevitably leads to mistakes.
To avoid those mistakes, automation helps make sure that you’re performing the exact test you need to, every time, without fail. By coding your tests into your software, you’ll be able to perform a number of QA tests and get reliable results back, without risking inadvertent mistakes resulting from monotony.
You Can Reach Scale, Without Scale
The final reason that automation is important in the world of QA tests — if you weren’t already convinced, that is — is that it can help to simulate an environment that you can’t produce manually.
The most common environmental circumstance you’d want to artificially test for is scale, where you want to test what’s going to happen with 10,000 simultaneous users. Obviously, it’s impractical to go out and hire a team of 10,000 QA testers for one test. That’s why most QA teams automate this sort of performance test, to help make sure they weed out scale-related bugs early, without having to actually release it to market.
But this equally applies to bandwidth constraints, device testing, or any number of other tests that you aren’t able to run on your personal computer. Anything that goes beyond your personal QA testing capabilities should be considered ripe ground for automated testing.
The Automation Balancing Question
Knowing when to automate and when to go manual all comes down to whether or not the automation will save you time. Your most precious resource is time, but while that may sound like it’s oversimplifying things it’s not.
Think of it in the short-term. If you spend one hour doing this every week and automating it will take 15 minutes, then automate it.
Unfortunately, things are rarely that clear.
Typically the picture is one where you aren’t sure exactly how long it’ll take to automate it, but you know that you’re spending a good 10% of your time doing one particular test. Should you automate or stick with the manual work?
The trick to this is that automation takes time and is an upfront cost. You want to be Time Profitable. That means you want to save more time (over the long run) than you’re investing in upfront.
Why Automation Should Start from Day One
We believe that automation should start from day one. This advice is most useful for startups who are just beginning to fill up their code repositories, but it’s worth reading through for everyone
From our point of view, there are four main reasons automation should start immediately, in parallel to your core development work.
1. Release New Features Faster
First and foremost, if you build in automation from the beginning, you’re going to be able to ship new features faster. That’s because you won’t have to spend so much time doing QA testing, which is a common bottleneck.
Shipping new features that are thoroughly tested also means less time fixing features that were released without being thoroughly tested. That saves you additional time, which means you can focus more on building those new features you were just thinking about. It’s a virtuous cycle that really pays off the longer you pay into it.
2. Don’t Get Stuck With Fear
When you don’t automate early, you get stuck with a question: what if you change something and it breaks? Fortunately, you don’t need to have that fear, since you’ve been automating things from the beginning — right?
When you automate early and automate often, you can immediately (and we mean immediately) see what the results of any changes will be on your production software. This means no longer living in a world of fear that one line of code being changed will kill your product.
3. Avoid Code Rot
Similar to getting stuck with fear, you also don’t want to get stuck with code rot. Every engineer is familiar with the phrase. It refers to code that is so old and so frequently changed without thought to the long-term consequences that it technically works, but it looks like a Rube Goldberg machine.
You don’t want that, the rest of the engineering team doesn’t want that, and the people who have to make changes to that code base three years from now really don’t want that.
When you automate tests, you force yourself to design your software architecture for the long-term, which coincidentally is exactly what you need to do to avoid code rot.
4. Automation as a Feature
Companies think of features as things that give them an advantage over their competition, but at the same time limit themselves in what they consider a feature. Automation is in and of itself something that can provide a competitive advantage.
That makes it a feature.
This is especially true if you are competing against companies that have completely ossified QA testing regimes. What we mean by this is that when you automate your code tests you can respond to the market faster, which means building more things people want, which in turn makes people interested in your company, which makes the voice of the market ever clearer.
Final Thoughts
There are plenty of preconceived notions about what automation is, how it works, and when it should be used. While we’re not dogmatic enough to answer those questions in a way that we expect everyone to live by, we do have one strong opinion: there is a time and a place for automation in every company.
The hard part is figuring out where that place is in your company. But as we’ve outlined above, there’s a clear path to figuring it out.