@Beaglesays

This post is brought to you through conversations recent, and past. The more recent ones tending towards LinkedIn discussions and reading the thoughts of others. Something that stands out to me is that many testers struggle to explain what they do when they test, why they do those things and even why they test. I see many testers justify their existence with “I find bugs”. Well that’s cool (to a degree) because testers need to find (important) bugs, but when you focus only on that, that’s underselling the craft and omitting many things that good testing brings to a company.

I remember many years ago being involved in an interview for a tester. Through the interview I counted 15 instances (I know I missed a few early ones) of the candidate saying “I catch bugs”. The candidates entire expression of testing was finding bugs, more nuanced discussions were largely beyond them. It’s a little sad when you realise this person had 5 years of testing experience. This interview might just have been a catalyst for me to make sure I could always talk about testing with some depth and clarity. It certainly encouraged me to reflect on and learn from my experiences and others.

To me, testing is a very broad church. To do it well you require multiple skills that are not limited to the domain in which you work or a couple of tools. Hell I use people and communication skills when testing that I learnt when I was a primary school teacher. I bring in to my testing things I experience as a cricket umpire (and I also wear my testers hat while umpiring).

As a tester you are working with not only hardware and software but also people. There are people that are in the same company as you and external to the company. Many of these external people are clients, others are providers of software that interface with what we build. To successfully understand the context in which I am testing I need to have sufficient knowledge of who the key people are, when I need to communicate with them and the most effective ways of providing them the information they need to make the decisions they need to make. In this space I’m managing technology, information, people and relationships and helping guide the production of quality software as part of a team effort. I might also point out that I’ve done all this and haven’t necessarily touched the software to be released (possibly not a line of code written yet). That doesn’t mean I’m not testing and it certainly doesn’t mean I’m not influencing what will be released or how we might go about building the code, documentation or release plans.

While I’m engaged in the above I’m constructing models of what will be built, trying reduce complexity and increase my understanding of how to best examine the software. I’m talking to people about my models (which generally have been physically represented either on paper, whiteboard or mind map) and seeing if they have the same models. It’s not unusual to find they don’t and this is a great test result. It shows that there are some different understanding and/or assumptions at play. Time to discuss, examine and course correct as required.

I like to talk to developers about the earliest point at which they can give me access to the code changes and what they can deliver to me. Give me something small and I can test it and give you feedback. How small – no single answer here except for “it depends”. I once had some code dropped to me that was “it’ll take you 10 minutes to test – at most, it doesn’t do a lot”. It did enough to show that there was a fundamental flaw in our approach. Most importantly we discovered this before it was a major strip down and overhaul.

Before I start testing I think about ways the software I’m about to test might be used, misused (intentionally and unintentionally), how I might make it “trip over itself”. How might I make the software do something that is “embarrassing”. How might I make the software hang, crash, respond poorly or simply behave like a 3 year old throwing a tantrum? Then it’s all “down hill”, right? Not really because it’s just getting started. The fun really starts now.

Now I have my my hands on the software (or some part of it). I have my preliminary test ideas and I’ll also include checking that any representations we have made about functionality can be demonstrated. Which is an activity that helps me build out more test ideas as well as adjusting or eliminating some of those I started with. To move to a slight tangent, this is just one reason why I personally dislike test scripts and the mindset of “that test passed”? What about everything else is going on around that single test, what is getting missed in the race for the “green tick”? At the risk of overemphasising the point, observation and concentration are key. I want to spot the unusual and unexpected.

I have yet to write that “I find bugs”. When I test I expect to find bugs but that is because I think deeply about how I might effectively test the software. In what ways can I test and find vulnerabilities? I’m not actually thinking “let’s find bugs?”. This might seem like a subtle difference (or perhaps not). This approach is explained by John Kay in his book Obliquity. If I say I’m going to find bugs, and make that my sole focus, I’m taking a very direct approach. I prefer an oblique approach where I focus on doing things that give me the opportunity to discover and learn and put myself in the best possible position to find important bugs. It also helps me focus on developing a solid understanding of various risks (because I need to convey this to stakeholders). Bugs are not my focus but an outcome of the way I test. This is just one reason why I don’t count bugs. The count, to a large degree, is irrelevant. There are many other dimensions of bugs that interest me far more before my (or another testers) personal tally. There are many other dimensions by which I judge my performance than by the number of bugs found.

If people wish to express their value in terms of “bug count” that’s a personal decision. In my sphere of influence (whatever that might be – and assuming I have one) I feel it is important to broaden the conversation. When I talk about testing I want to amplify the skills required, the depth of thinking and analyses needed to do it well. The broad experiences and sheer hard work that feed into decision making, the people skills, the reporting skills, collaborating, influencing and helping people and products improve. That list is just a starting point, far from exhaustive. In that conversation I want people to appreciate that bugs are found because good testers make it inevitable they will be found rather than testers only find bugs. There’s a huge difference in value statement between the two.

I would much rather have the conversation about why I find bugs that others might not find rather than bug numbers. I would like to be recognised for more than just finding bugs, because, frankly, I (and many other testers) bring way more than that to the table (if you’re reading this and don’t know any of those “other testers” get in touch with me and I’ll send you some names). I would like to move the focus of testing discussion to the nuances, challenges and difficulties of good testing and how testers can help with quality (and that testers don’t own quality or assure it). I really want to see more testers thinking about, and communicating with clarity, what it is that makes testing valuable. Elevating the status of testing by sending clear and accurate messages – well, it’s in our hands.

Cheers

Paul