Selecting a testing tool (4)

Continuing from our earlier posts that looked at the common reasoning behind the question of whether to build or buy testing tools, let us now look at -

Reason 4 to build a testing tool in-house - We should build it since we just need a simple solution which our engineers should be able to quickly put together; our software testing requirements aren't too complex; we know what we need and figure it should not take much time or effort to build a testing tool

Organizations find it tempting to put together or build simple test tools to address a specific problem or area without considering, at the risk of making a cliched statement, the larger picture. If your test tool is not easily enhance-able / maintain-able and you are limited to the functionality you built-in initially, you can be sure that sooner than later, your needs will outgrow what your test tool offers - the test tool will soon hinder than aid your efforts. I have seen various groups create such “simple” tools to address specific pain points. Over time, various such “simple” tools were created and put to use by different groups. Since little thought was given to integrating the various tools or making them work together, the proliferation of these “simple & standalone” tools posed a significant handicap to the organization's software testing efforts.

From experience, such test tools tend to get discarded after some amount of usage and a suitable replacement is found. However, there are cases where these tools continue to be used – the organization has already invested significantly in the existing test tool and may not be willing to incur the additional costs associated with a migration to a new tool.  In either case, the organization has already incurred costs on the simple tool and will incur more costs to procure and move to a replacement tool. It would be better to consider the larger picture when making tool decisions. The test software (testing tools) should enable your organization to perform better software testing. Look beyond the confines of your group; examine (best) practices from other groups and tools they use; explore any possible changes to your current processes / additional requirements for the tool you intend to build and consider if the test tool can handle them.

It need not just be new or changed requirements that can make your tool redundant. Simple scalability issues can cause your simple tool to pose complex issues. If your tool is designed to handle the needs of your current group, consider if the same tool can handle the requirements of a larger group - will your tool scale or crawl ?

Does your tool consider all audiences who might potentially use / want / need to use it ? Some tools are created to solve pain points that a section of engineers may encounter. Their usage, output and reporting are often put together to provide the basic information that those engineers would need. However, when the same tool needs to provide information to a wider audience or someone in the business side of things, a different usage and reporting paradigm emerges. If the tool cannot meet the needs of the different audiences - current as well as potential future ones, it can again be a drag. What if the products you are testing changes significantly so that the tool also needs to be changed to handle newer operational / functional needs ?  Can your tool adapt to changes ?

We'll continue looking some more at the build versus buy choice for testing tools in this series of posts. Also, coming up is a comprehensive article on making testing tool decisions, listing some questions you need to ask and evaluate before making your decision of whether to build or buy a tool for your testing needs. Stay tuned. 

Image courtesy: Danilo Rizzuti /