A Live Developer Journal

Orange spiral bound rocketbook notebook surrounded by bits and bobs like a happy fabric pretzel and tea boxes.

Thoughts On Measuring Software Quality

When you think of the word “high quality”, what comes to mind? As I wrote that to you, I looked at the notebook on my desk.

It’s an endlessly reusable notebook, and it’s my all time favorite purchase under $100 for the following reasons:

If I were to rate the quality of this notebook, I’d give it 6 out of 5 stars. I wouldn’t change anything about it. In fact, I am mildly offended when I read Amazon reviews about it that are less than 5 stars. That being said, the reviews cover some really good points, and demonstrate that a product can meet some people’s needs whilst falling short of meeting other people’s needs. Some examples:

Asides from practically waving the Rocketbook at you, this story has implications for how the tech world tends to think about and measure software quality.

In the tech world, software quality is often described it terms of how well a software project does what the requirements says it should do.

None of the reviews above, good or bad, rated the quality of the product in terms of how well it conformed to the product description. No one wrote “Yep, this product is exactly what the product description described. It DOES have 36 pages. You CAN wipe it clean with a damp cloth and you CAN save your notes as PDFs or JPGs. 5 stars!” Instead, the reviews rated the quality of the product in terms of how well it let them to do the things that mattered to them.

So why do we rate the quality of our software like this? Why does our measuring stick start from the ‘solution’ level, instead of the problem or person level?

Measuring the quality of our software by the number of defects it has, the number of features, cost, time, best practices, performance etc tells us nothing about why we’re trying to do these things, and so it’s difficult to prioritize, eliminate or make thoughtful trade-offs between them.

Knowing what matters to people and why, allows us to make better decisions down the line when it comes to making trade-offs that impact customers, teams and organizations.

So, whenever you see/hear a requirement or a quality statement, ask yourself and your team the following questions:

P.S. If you get a chance, read the Quality Software Series by Gerald Weinberg, it inspired this article. Some books are gold mines in that there are a few great nuggets. Other books are coal mines, where every shovel has something useful. As one of the reviewers in the preface says, this series is a bunch of coal mines. It amazes me how the very best ideas about software come from the oldest software books, here’s to hoping they become mainstream.