Imagine the scene. Its a hot July day in Leeds in 2011. We are in the old Agfa building on the outskirts of town. Mostly no longer in use, operations outsourced long ago. One of the conference rooms was in use that day though. Michael Bolton was delivering Rapid Software Testing and I was very excited to be there. There was a mix of practicing testers and test managers there, so a heady mix.
Pretty early on in the first morning, talked turned to test estimation. Michael said words to the effect of:
“If someone asks how long it will take to test, ask them when they want it by. You can do any amount of testing, just keep talking and give good information about what you are up to.”
Michael Bolton (Paraphrased)
This seemed sensible to me as I had been involved in lots of circular sessions about estimating how much testing could or should be done. What Michael was saying seemed a positive way of saying we can do some testing, in a sensible way which is sensitive to your delivery needs. Especially after being subjected to very strange spreadsheet systems with weightings and formulae trying to crack that particular code. To everyone else though, this was incidendiary stuff. The rest of the morning was spent incredulously discussing test estimation and how to fight for the time needed to test. Eventually a truce was called and we got on with a very engaging course.
Ever since then I generally haven’t given a hoot about test estimation. In fact, I would say that its the ultimate in distraction for testers and divisive to acheiving any form of team testing culture. Anyhow, testing always got squeezed so it seemed like a forlorn hope to me. Why not have an answer which sets you free to actually perform the activity of testing? Plus gives people a warm feeling about how the tester isn’t getting in the way again? Test estimation always felt like being asked to count the number of sweets in the jar. Where the jar changes shape and volume continuously. And no one asks what do you mean by jar and why do you need a jar. Or sweets.
If you hear these classics:
- Thats three points effort for dev and how many for test?
- Its a one line change but effects the whole system. How much testing will it need?
- How much time shall we allow for bugs?
- I’ve refactored (code for usually rewritten without tests) this whole component. It will need regression testing. How long will that take?
- No one really knows how this works, but we need to make a change. How long will you need to test it?
Then you are deep in the wrong conversation and need to drag yourself out of there. Test estimation is the wrong question with even wronger answers. It is most definitely a trap. Being a fundamentalist about it doesn’t help either though and I can be reasonable at times. A very good Agile Coach I know plays it like this. It’s a long game at most organisations and the practices that got you there won’t be changed overnight. When someone asks you for X, give them a sensible version of it, but use it as an opportunity to educate. That might be gantt charts, adding more people to something thats already late, increasing work in progress or whatever. Same for test estimation.
So, when faced with the situation above, I might respond with something like this, before pressing the test estimation button:
- Three points is fine as we will test continuously together, not only at the end. This may provoke another discussion about the scope of the ticket, which is cool. Otherwise it means I back your sizing and will test with you all along the way.
- Testing alone is not enough to solve this problem. One line changes that impact the whole system mean that your architectures needs work, we don’t really know what the risk is and maybe some kind of sensible toggled or stepped release is need.
- I just say I don’t know, because I don’t. If there is a worry about this, advocate for building smaller slices of the feature. Then provide test ideas and support to those who are doing the programming. See the point that Michael makes about keeping talking and providing good information.
- I hold back from the usual lecture about what is rewriting and refactoring and switch the focus to unit and integration tests first and foremost. If there are none, then the conversation probably won’t be about test estimates anymore.
- Finally, an ancient system, original programmers are long gone, it does a thing and no one dares touch it. But touch it we must. This is where you can shine. Say that before you estimate the time it will take to test the change, you wan’t a day to explore the system. I guarantee that what you find will flip the conversation on its head. How long will it take to test will often become maybe we should consider our options here.
Organisations will still ask for test estimates sometimes, I get that. Usually part of a wider dysfunction. I want maximise time spent performing testing. I want to work with teams to give continuous feedback. Asking me to estimate testing as some kind of separate entity to the teams work is a wedge between roles we don’t need. I enjoy my freedom from the tyranny of estimating testing. It’s made me a better tester.
Diagram Industries 