A question that started in the ABTesting slack group and has found its way to LinkedIn was wondering about whether an unethical product, even if exquisitely built, can be referred to as "high-quality".
This question resonated with a biff I have with the term "quality" for quite a while now. Specifically, with how it is commonly used. Since many of us have the word "quality" in our role description, many of us and of our colleagues in other software specialization are using it to indicate the sum of all desired things we might want in a piece of software. As we do that, we are shooting ourselves in our metaphoric leg. How bad is it? well, as I was writing this post, I stumbled upon this episode of the testing peers where we can hear four professionals who care about their craft, spend half an hour talking nonsense. By the end of the episode titled "what is testing" they do recognize the fact that they have not actually defined it.It is of no surprise, this term is difficult to define and is pretty vague, and as I've written in the past, it does not conform with the idea of quality in other fields. When my manger says "we have a quality problem in our product" he means one thing that is probably not the same as the product manager saying "We need to have this shipped with perfect quality", and is surely different from the now-so-common definition of "value to some person who matters" or the approaches adopted by ISO - "fitness for use" or "customer satisfaction". What we are getting when we use such an ambiguous term is a feeling of consensus that will blow up in our faces once the actual differences will start to show.
Moreover, "improving quality" is not actionable. If you actually have such a problem, you have a lot of work to do just to clarify what you actually need, and you will need to override people instinctive understanding of this term. Some people would go as far as claiming that "quality" is just about technical mastery and skillful coding. Others will stress the visibility of the product and others yet will look at the extent in which it gets the job done. All of those are important, but we need to align in order to move forward
Another notion I've stumbled upon sometime during my career is that "quality" is just another word fro "property", which leads me to suggest talking about the specific qualities we are actually seeking in our software. It could end up being a long list, but it will enable us to align more easily, and to break the dichotomous view of "high\low overall quality" - Software (and life in general) is all about choosing the right tradeoffs, and I might go for something that looks nicer even if it is not as fast, or choose the product that preserves my privacy better over the one with more features and better integrations - we need to use a language that helps us notice these possible tradeoffs instead of concealing them under the very broad term "quality".
If "Quality" is a profane word that should not be used, what should we talk about instead? My preferred way is to break it down to different qualities, and preferably, to avoid confusion: Desired properties. What sort of desired properties are there of a software? I'll list a few here, and there are probably more that would be relevant in the various contexts one might find themselves in:
- Correctness: This is the extent to which the software is doing what it is supposed to do. A tax calculator makes no tax mistakes given the proper input, a game is responding to the user commands in the expected way, etc.
- Robustness: The product can withstand messy (and even malicious) input, recover from mistakes and does so for a prolonged time.
- Scalability: The product can support the expected growth in a viable way.
- Performance: Personally I lump here both the quickness of response and the resources consumed by the system, those can be separated if we are in a place to make internal trade-off (for instance, throw more hardware to get faster results without a redesign).
- Privacy: How well is the product protecting human data that it handles (or generates).
- Ease-of-use: How well can people achieve their goals with the product.
- Attractiveness: How appealing is a product to the target audience, that would conform to what James Bach has named "charisma"
- Explainability: The extent to which the behavior of the software is predictable or easy to explain.
- visibility: can we figure out what happens in production? can we use that data to improve the product?
- Safety: will the product kill someone? Harm them in any way? will releasing a new version put us in risk we can't tolerate?
This is a partial list that can be extended with a lot of other properties (an example can be found in the paper by Rikard Edgren, Henrik Emilsson and Martin Jansson extending James Bach's quality criteria), but one obvious thing is not on my list: Testability. Why is that? because in itself, it provides very little value. I think of testability as a secondary desired property that helps achieving the others or verify their state.Other properties in this category could be maintainability or extendability I also don't consider process properties in this list, even though they are often more desirable than almost every bullet in my list. Things like efficiency or pace of deployment: I had to draw the line somewhere and so I chose to stick with properties that are attached to the product and not to the teams building and maintaining it.
So, next time someone asks you how to improve the quality of your product, take the time to understand what do they really mean. Are they bothered by functional problems? do we have a security problem? There's no need to go out right and tell them "don't use this meaningless, confusing word", but try to drill down using guiding questions such as "what is the most significant current concrete pain that we feel now?" or "do we want to invest now our efforts in increasing our ability to spot problems in production before our customers, or do we have more pressing matters?" get to somewhere concrete and work from there. Ask such questions again after there has been some improvements in the top priority desired properties.
