When business owners discuss the best way to develop software, testing is always a scary topic. How much time and money you should spend on software testing is not an easy question to answer. What type of human resources should you commit to testing is an even harder one.
Test automation requires some programming skills, but software developers are a precious resource. Why waste developer time automating tests when you have to launch new products and features?
But, bringing on a dedicated test automation developer is expensive. You could probably hire two or three manual testers for about the same amount. What makes a test automation engineer that much more valuable?
These questions and their answers mean everything when creating a software testing strategy.
We recognize and advocate the benefits of both manual and automated testing in their proper contexts. We understand that the business must weigh the benefits of each testing type to create a plan that returns the best ROI.
The key to creating the right test plan is when your developers or manual testers ask for more help, listen carefully to their problems. Knowing what they have trouble accomplishing will help you discern what type of help they need.
Do you need a manual software tester?
There is one simple rule in testing any software: someone besides the developers that made the software should test it before release.
Your developers know how their software works. They test it to make sure it works that way. But developers have different skills and viewpoints than uses do on how their software works. Consequently, they often struggle identifying issues users will have with the product. Only an external set of eyes can give developers a critical early view of how users will react to the software.
Which type of tester to use depends on the kind of product being tested. If it is a customer facing application, such as a web or native mobile app, use a manual tester, ideally one without development skills. If it is a programming interface that will only other computer programs will use, you will need someone with development skills.
Once you get people besides developers testing software, you need to make sure they are pulling their weight by asking these questions:
- How well do your developers say the testers communicate bugs and other problems? What is the average time for developers to work on a specific tester’s bug reports before they are fixed? Writing a concise bug report that points the developer to the root of the problem is a learned skill. If they are not performing this task well, consider enrolling them in a class or pointing them to test training literature.
- Has there been a drop in bugs reported out in the field since this tester came on board? If your testers are not providing a quality view into how customers will react to the product, perhaps they should spend some time in the helpdesk area to get a better idea of how end users issues.
How much automation do you need?
Without automated testing, software wouldn’t function. The real questions you need to ask is how much testing should you automate. Here are some things to consider that can indicate how much automation is missing:
- Is the alpha testing environment unreliable for basic testing? Users expect quality enhancements/fixes in every release, and testers will expect a certain level too. When developers are not delivering working code, this is a red flag that quality unit testing is missing. Good developers always build test automation for unit tests!
- Are your manual testers complaining about the amount of time it takes to verify that already-tested functionality has not broken? Are they managing long manual test scripts to remember the critical functionality to retest in each release? Almost all manual testing scripts can be automated. The longer they are, the more likely it is you need to consider automation to speed up the process.
- Is your release process slow and error prone? For new developers who have never released software before, they may have no idea just how much of the process can be automated.
Continuous Integration (CI) can drastically lower the amount of time it takes to release and increase confidence in the ability to release software more often. The proper release strategy can reduce release schedules from once or twice a year down to weekly. Some companies release new software every day. But don’t you dare expect any team without significant automated processes in place to deliver that kind of turnaround and maintain consistent quality. It is simply not practical.
Who should own your software testing strategy?
If you decide to increase automated testing, who do you assign to the task, and what are their priorities?
To begin, it should be a skilled developer. If you expect the best ROI, you need an architect. Manual testers simply will not have the skills to create something that will last. It’s a good idea for you to at least temporarily bring in someone who has experience setting up good CI environments and coaching your entire team on how to keep it rolling.
Then find a developer who has shown an interest in automated testing. Make sure they are capable of delivering on real projects and that their interest isn’t just theoretical.
Are you prepared to make this a priority?
Once you’ve selected a first tester, they should work with fellow developers to start automated testing.
Here’s why that investment is worth it:
- Developers, not testers, should be primarily responsible for the quality of their software. Insisting that they consistently write unit tests alongside their new and modified code is the best way to get them working true quality into the foundation of your product.
- With the right tools and testing structures, unit tests run much faster than integration tests. This means your developers are much more likely to find a problem before they shift their focus to other tasks, which is critical for productivity.
- Unit tests are also easier to write and maintain. It’s best to start out simple, testing small bits of the code at a time. This is where someone with experience in mocking technologies can help. Once they have a handle on that, they can add more complex integration tests.
- Unit tests are developers’ best documentation for other developers. Your developers are eventually going to hand their code off to someone else. Automated unit tests force other developers making changes to first consider all of the different angles the original developer had to consider when writing and enhancing the code.
- Automating the manual tester’s regression scripts first, especially without the aid of the developers, is a practice doomed to failure. Once your test automation expert moves on, the other developers are much less likely to maintain their work, and it is unlikely that the manual testers will have gained sufficient ability to do it themselves.
Your developers should not try to set up unit tests to cover all, or even most, of their legacy code. While it may not be unit tested, it has probably been extensively manually tested. Until it gets changed in some way, there is less of a chance that it will break. It’s enough for the developers to set up their code scanners to ensure that new and changed code is always unit tested.
Once you’ve given your developers the tools to automate unit and integration tests for their code, you can build the infrastructure for manual testers to automate those scripts they’ve been manually exercising.
Unit and integration tests help testers better trust developers. New testing tools allow them to automate the boring parts of their job. Now they can focus on the indispensable (and fun) part: testing sleek new functionality and giving feedback (including new automated tests) to make it the best product your customers have ever seen. That leads to a solid product that will last for a long time to come.
