Thursday, June 17, 2021

Is all testing context driven?


So, AST is crowdsourcing a book on testing, and I was reminded of the current question by James Thomas's response (more details about it in his blog). So, right before reading his answer, I went to the AST slack channel and posted my own response.

As this blog has been quite dormant for a while, despite my best intentions, I thought it would be a good start to post my answer here as well. 
Who knows, I might even get to translate it to Hebrew later. 

After reading James's answer, it seems that we say roughly the same thing, only he's wording it better and more precisely. We both make a point about being context driven.


So, without any further hassle, here's my response:

All testing is driven by context, some of the testing is doing this explicitly.

If we face the truth of it, context provides a set of limitations and pressures that affect the way our testing is done and the extent of thoroughness one feels they can and need to examine an application. In that sense, context is driving everything.
A more precise look on the state of the industry would make the distinction between context driven and context dragged, the main difference would be the place that context plays in the way we talk about our testing. For example, in a context driven situation, one might say "We're in a fast-paced team, if something goes horribly wrong the worse thing that can happen is that the company will lose some money, which is gained multiple times over by releasing quickly, so we devise a strategy that will lay out decent automated checks, but do deep exploration after the product is deployed. We still need to improve on exploring edge cases while defining features to reduce the noise coming back from such exploration"
In a context-dragged situation we might say "Testing in production is the way to go forward! We deploy faster, getting important things fixed and increased collaboration within our team." and would strive to do the same in a more risk averse environment (say - testing a self driving car algorithm).
There's also a third, very common type of testing, which we might call "poor testing", those are normally context-oblivious, and uses the process they know and the test level they are familiar with without considering the business needs. Naturally, it's not a very interesting type of testing to discuss