TLDR: Rather than discuss ROI in depth, I want to explore how to evaluate ROI for yourself, so I provide some questions as an exercise e.g. “Why do you use ROI?”, “What do you gain?”, “What do you lose?”, etc.
I am often asked about ROI. And I wrote a parody answer. In this post I explore initial beliefs and then questions to explore and expand my model of ROI.
“How do you calculate ROI of X?”
Where X is “automation”, “this tool”, “exploratory testing”, “blah blah blah”
And the answer is that I don’t.
I wrote a parody blog post “How to Invest In Testing” and one section of that post basically makes fun of ROI calculations.
I don’t work with tools or decisions that justify an ROI. I don’t see a lot of value in it so for me its an easy parody target.
I think people overuse ROI. And I’m cynical in the sense that I think people use it without exploring alternatives.
Bureaucracy loves ROI. Because process, numbers, calculation, busywork.
Consultancies love ROI: calculators, processes. You can’t make a decision.
Check out this web search Google - Test Automation ROI Calculator.
How many test consultancies come up?
ROI is often used for a sales process. Why is that?
First Principles and beliefs
Rather than getting sucked in to ROI. Let’s look at first principles.
I don’t use ROI, why?
- I see no value in it for the work I do,
- I don’t have to
- I have other ways of making decisions
We could investigate that.
I see no value in it for the work I do
The costs I work with are almost negligible compared with the costs of the business or the projects.
I think ROI works well when deciding on projects go/no go. Rather than decisions on projects.
I don’t have to
This is probably the most important answer.
If you are working in an environment and you don’t have to do an ROI.
Then don’t.
I’m willing to bet that the ROI calculation does not have a numeric representation of all the factors in your environment that are important. e.g. HS2 ROI Report
The HS2 ROI Report requires selling the infrastructure to reap a 50% return on investment) not a great return and it ignores a lot of cost. This post was written in 2011 when the estimated cost as 14bn, in 2018 the current estimated cost is between 60 and 100 billion independent inews (I can’t tell if the ROI has changed in response to the estimated costs).
In the formula you use for ROI, what is your number for:
- job satisfaction
- opportunity cost of delay due to using an ROI calculation
- communication mismatch with other team members
- etc.
The formula doesn’t cover everything that might be important.
If you are working in an environment that forces you to create an ROI then you’ll probably end up having to game the ROI calculation to get the result you want anyway. So you’ll need to have other ways of making decisions.
I’m not a consultancy that tries to sell big services based on an ROI calculation.
I have other ways of making decisions
- Make the decision
- Weigh up options, make the decision
- decide on something, get the budget to implement
An evaluation criteria I use - “can I justify this decision?”
Things that help:
- Tossing a coin
- Cost/Benefit/Feature Analysis
- Comparative Options Analysis
Why toss a coin? If you get a result you don’t like, you feel it, and don’t always go with what the coin says.
Some people equate Cost Benefit analysis with ROI. I think they are different.
But rather than try and argue and engage on the level of numbers and ROI. Why don’t we consider our own positions on ROI and what they are based on.
I’m going to give you an exercise, and you can do this if you are Pro or Against ROI.
I’ve done the exercise, I found it very useful. I’m not going to cover my answer here as it would be too long, but I’ll create a blog post and a Patreon video covering it.
Exercise: Ask yourself questions about ROI
ROI is often a default reaction. I encourage you to think it through for yourself.
- Why are you using ROI?
- What is your definition of ROI? Does that match the rest of the world’s?
- What do you gain by using ROI?
- What do you lose by using ROI?
- What would happen if you did not use ROI?
- How could you make a decision without using ROI?
- How does using ROI help you make a decision?
- Do you always use ROI? What was different for the situations when you didn’t?
- What other questions could you ask yourself about ROI?
And if you don’t use ROI, try these:
- Why are you not using ROI?
- What is your definition of ROI? Does that match the rest of the world’s?
- What do you gain by not using ROI?
- What do you lose by not using ROI?
- What would happen if you did use ROI?
- How could you make a decision using ROI?
- Do you never use ROI? What was different for the situations when you did?
- What other questions could you ask yourself about ROI?
I worked through this exercise and found it useful. I will expand on my answers in Patreon, but I hope you perform the exercise yourself, and I’d be interested in what you find out when you do.
