Selecting a Test Tool

Test tool is a broad term that encompasses test automation tools and other tools used as part of testing. These could be defect tracking tools, reporting tools or test management tools. When an organization or a group within a larger organization decides that it needs a test tool, a likely question that can crop up would be – can we build it in-house or do we buy a commercial tool ?

For some groups, the decision's already made. There might be a centralized entity that has ordained that a particular tool or framework be used and all other groups simply toe the line. But what would you do if you are making a fresh choice ? Would you have to face the above question of build vs buy ?

Lets look at a few reasons why organizations might make either choices – build the tool in-house or buy the tool. First, why would organizations choose to build a tool.

Reason 1 – Short term solution

We just need something for the short term, a temporary solution, something quick for the time being. We are hard pressed for time and cannot spend time evaluating a lot of choices. Let us quickly put together something now and later explore more longer term solutions. Any of these sound familiar ? Looking for short term solutions ?

We must realize that when we decide to build something, even if it is supposed to be a temporary solution, we will be investing money, time and resources to create, use and maintain the tool.

The tools we employ to facilitate our testing, can soon mould our work habits around them. Sooner than later, the tool will become a part of your regular job and exercise a certain degree of inertia towards any change. When the tool falls short in certain areas, you will (have to) come up with workarounds to continue using the tool and over time it becomes harder to change established work patterns and tools learnt.

In most cases, once employed, a tool becomes hard to change. Lets take the example of an automation tool / harness that your in-house resources put together. Once you have reached a certain level of automation using the tool, how easy will it be to scrap the tool and opt for a new one ? If your tool is lacking in certain features, would you spend the time and effort to enhance the existing tool or would you ditch it to go for an external tool that offers those extra features ?

In my experience, I have used tools that were home-grown and met the basic requirements of our group. However, the tool paled in comparison to some of the externally available tools in terms of features and options available. Since we had invested enough time and resources on the tool, it became hard to let go or even migrate to another tool without incurring significant additional investments of money, time and resources – which we were reluctant to do.

In essence, you cannot afford to justify a decision based on it's perceived or assumed short-term need. Every decision carries a set of ramifications with it and it is rare that you can easily back out if things do not work out as expected. We'll explore more on the question of build vs buy in subsequent posts. Stay tuned and send in your feedback using the "Contact" link at the top of this blog.

Image courtesy: Simon Howden /