TestingCup is getting closer and closer… The program of 2019 Conference has been all chosen and designed by the Committee headed by James Lyndsay.
After TestingCup, James will also run a 2-day practical workshop „Insights into Exploratory Testing”.
Taking advantage of the cooperation with such an experienced Specialist, our editorial team decided to ask James a few questions.
TestingCup: Do you agree that all testing is exploratory, so there is no need to use a phrase „exploratory testing”?
James: Testing, for me, involves comparing and judging. If you’re not able to make some kind of comparison, and you’re not informed-enough to make a judgement, then you’re doing something else.
Exploration is more about finding, and choosing, a comparison – and about building one’s judgement. It’s a fundamental skill. I’m shy of definitive statements like „all testing is exploratory”; I’ve not done all the testing…
Testers need to find the truth about their systems – particularly about behaviours which are unexpected, unwanted and accessible. The sooner they find that information, and the more they tune their search to business and user needs, the better. I use the term “exploratory testing” because it’s broadly understood, in my community of testers, to mean looking for trouble rather than verifying capability.
TC: What metrics can exploratory testing provide?
J: My most basic metric is time; budgeting the time we might spend looking for trouble, then measuring time we have used. We can look at how that measurement changes as we, and our systems, change. We might split it by system area or by testing approach, showing us what we’re paying attention to, where we’ve been blocked or when we have needed to change our focus. If we see time budgeted and not spent, or an area that has no budget, or work which is taking all the available time, then those insights help us to ask the right questions about our work, and its effect on our products. I also ask testers, once they’ve explored, to consider how “done” they think they are, to give a broad metric for coverage.
Metrics condense broad new knowledge into data points, making assumptions about size or completeness to allow aggregation and to show trends. Some information can’t be summarised in this way while keeping much meaning. For instance, an exploring tester may reveal real problems, highlight risks, and help the project understand the scope of its uncertainty. I’ve not found a way to squish that useful, detailed information into simple metrics.
TC: Which method(s) for managing exploratory testing (besides SBTM) do you recommend? Please tell us a few words about them.
J: Software systems are more invented than built – even when integrating known elements, the configuration, plumbing, data and use tend to be both novel and crucially influential. Exploratory testing is at the sharp end of this experimental, incremental, discovery-led work. I recommend that we manage this by aiming to get the best from a given project budget, not by setting estimates and working towards pre-selected product goals.
Finding the unexpected is a gamble with limited resources, wide options and emergent “rules”. Small, parallel investments help to work out where to place one’s bets, how to shorten the odds. With careful gambling, one can discover which constraints are real, put more into productive paths, take useful shortcuts, and be more aware of reducing returns. All of this helps you steer your team’s work throughout your budgeted time.
When looking at overall strategy, I reread Henry Mintzberg, particularly thinking of what he calls “learning strategy”, and when thinking about how one might manage an exploratory project, I go back to papers by Sylvain Lenfle.
TC: Poland – what are the three first things that come to your mind?
J: Beer, pierogi, and how to pronounce the letter Ł. The Hanseatic League, the Second World War, how Poland’s long green fields look from the air. Or, perhaps, I think of my Polish friends in London; Kasia, who can turn her hand to anything and always makes it exceptional, Jarek who treats the world with deep insight and gentle humour, and Jolanta who sees every child in her classroom as a particular and talented individual.
TC: Thank you.
Don’t miss the chance to learn from the best and most experienced Experts!
[info]Early Birds tickets to James’s workshop are still available until 14/05/2019!
Register at the Participant Zone >>[/info]