As both a consultant specializing in software quality and a maker of test automation tooling, I speak to dozens of software teams every month about their challenges in delivering consistently high-quality software at an ever-increasing pace.
More and more teams recognize the need for an effective full-cycle test strategy and have selected great tools and techniques, which is awesome! Even so, a few seemingly small misconceptions about how to apply these tools and techniques often prevents teams from gaining the benefits from them, and that’s not so awesome!
It’s kind of like buying a great new jacket, only to then try and wear it back-to-front. It’s not only awkward, but you also can’t see where you are going.
As silly as this analogy may seem, I often see software teams miss out on the potential benefits of quality tools and practices by applying them back-to-front.
When test automation is back-to-front
To understand the front and the back of testing, you have to take a moment to consider what it means to be the right way round. And that means starting with considering what the role of automated testing really is.
The role of automated testing is to ensure that your intended value is being delivered at the intended quality.
Let’s take a pause to think that through before we dive into more detail…
What are Value and Quality?
I’ve written in detail about these concepts before, and as they are extremely difficult to define, I am always evolving my thinking and definitions, which are currently as follows:
- Value is the reason you are building your software in the first place. It is typically expressed as a problem it solves for your customers or users. These customers or users are the consumers of a solution which ultimately benefits them. And because these benefits are subjective, it means value is based on the perception of the consumer. Therefore I define:
Value is a benefit, as perceived by the consumer.
- All value starts as “Potential Value”, meaning that it has no benefit unless a person actually receives it and derives a benefit from it. To achieve the potential of the value, you apply quality functions as you design and build the value. The more effort towards quality that you apply, the closer you get to achieving the full potential of the value. And since quality is extremely subjective, it is also based on the perception of the consumer. Therefore I define:
Quality is the extent of the potential value realized, as perceived by the consumer.
- The mission of the team to deliver the potential value requires constant readjustments that are continually applied throughout the product development life cycle. These must not be left until the end or owned by any single person or department.
That last point is the most critical. Only if quality functions are applied from the very beginning, throughout the life cycle and by everyone, can they extend the potential of your realized value.
To only apply quality control at the end of the process is backwards, and here is why:
As many of you may know, the cost of a bug is exponentially greater the later you discover it, and it also takes longer to fix it. Therefore only applying quality functions at the end is to apply them in the most expensive way.
But what you might not know is that more than half of all software bugs are actually created before any software is written at all — due to missing or incorrect specifications.
In other words, most teams skip focusing on quality at the point that it is most important and most cost-effective, and they instead concentrate efforts when it is too late and exponentially more expensive.
To some readers, this might sound strange, but to some others, it might sound strangely familiar!
Don’t let the four phases on the diagram fool you into thinking this is only a problem in waterfall delivery models. The same is true for any iterative approach. Whether you’re writing a story or a project plan, applying quality at the end is back-to-front!
Here are some signs that your team might be putting your quality on back-to-front:
- You have a QA role or department that “owns” quality
- You have little or no test automation and rely on manual/human regression testing (including Rainforest QA or an army of testers)
- You hired a test automation engineer, but they are the only ones writing any automated tests that run after developers commit code
- Your engineers don’t do their own automated testing and rely on QA to find bugs
- Your overall automated test coverage is low or is comprised entirely of unit tests
While all of these are much much better than having no quality measures at all, it is like wearing your raincoat back-to-front. It does still provide protection from rain, but it quickly becomes cumbersome and stops you from being able to continually adjust your path to get to where you’re going.
How to apply quality the right way round
The process of actualizing the value of your product is a constant process of adaption and readjustment. Think of it like doing the Tango- gracefully dancing with your partner, staying closely in step with one other as you glide across the dance floor.
I call these steps Quality Functions, and I’ve compared them to the actions of care you apply to your product. Now, in response to popular demand, I have produced educational content that explains each of these, and will shortly be available in my forthcoming book, “Quality, Faster.”
Check out these synopses that explain how you can be sure to apply quality from the front:
- Ensure quality is everyone’s responsibility
Given that the most critical point to apply quality is early in the process when the design and business teams are involved, the entire team must buy in to their responsibility to quality. This is not always easy to achieve, especially when there has been a legacy of quality being left to the QA department. So, in this section I share practical advice for how to ensure full team ownership.
- Ensure quality starts early in the definition phase
There are only three types of bugs, and two of the three types are caused by poor specification. If I could recommend you take only one action to increase the quality of your software, it would be to unite your Business, Product, UX, Development and QA teams around creating a shared communication of the expected value and quality. Whether it be delivered as a whiteboard session or collaborative executable specification sessions, the communication between parties is in itself the most valuable quality function of all.
- Bake value into your specifications
Given that the role of automated testing is to ensure that your intended value is being delivered as specified, a clear definition of the value must be present within your specifications. This means communicating not just the solution, but also the consumer and the benefit they receive. The following story narrative is great at doing this and takes on this structure: “In order to <benefit>, As a <consumer>, I want <a solution that provides the benefit>”. And by adopting a “Given, When, Then” scenario description instead of just bullet-point acceptance criteria that tie right back to the story narrative, you focus the whole team on ensuring real value is always delivered, while also freeing your developers to recommend the best solution to achieve this.
- Generate constant feedback from the core
Many development teams have already experienced the benefit of real-time feedback from unit tests while they create code. What is less well-known is how to receive real-time feedback on core value and end-to-end value tests as well. When you do this, it ensures that your product never deviates from delivering the intended core value, and your team is always able to course correct early and cheaply.
- Free your QA team to play to their strengths
Test automation is great for efficiently testing what is expected to happen and not happen. But what about the bugs that occur because nobody could foresee certain scenarios? Your QA engineers have a special superpower for anticipating the unexpected, and they should be enabled to spend the majority of their time doing so. If you successfully use test automation to automate the expected, your QA engineers can save you from the unexpected.
- At this time in 2016, there is a plethora of tools, techniques and processes that can help deliver higher quality software faster than ever before. But to do so, we need to accept that delivering high quality value requires a constant ongoing tango of quality functions applied from the very outset of the idea, through to the end delivery of the value to the consumer.
- The most important and cost-effective place to start applying quality measures is from the very beginning and then throughout the process. To only apply quality at the end of the process is to be completely back-to-front.
Thanks, and I look forward to helping you deliver higher quality software, faster.
Let us help you on your journey to Quality Faster
We at Xolv.io specialize in helping our clients get more for less. We can get you to the holy grail of continuous deployment where every commit can go to production — and yes, even for large enterprises.
Here are the things we offer:
Services (we help you)
- Software Development
We build mobile and web applications for you quickly that stand the test of time. We practice what we preach and have a very high quality standard.
- Digital Modernization, Coaching, and Consulting
We’ve helped clients such as VW and Audi improve their software delivery processes manifold by working alongside their teams and agencies.
Let us power up your teams skills so they are armed with the know-how to produce higher quality software at higher speed.
Products (you help yourself)
A single source of truth for specs that allows your team to collaboratively write specifications, which improves velocity, decreases bugs, and increases quality.
- Quality Faster
Gives your team the knowledge and skills they need to practice our teachings with our guide.
Have something else in mind? Get in touch.
Federated Design Systems with Storybook
An exploration into how Storybook-based design systems can be used to distribute component code.Sam HatoumMay 7, 20206 min read
How to Design and Build a Hyper-Fast Test Automation Stack
Why you need real-time end-to-end test automation, and advice for the best products to create it.Sam HatoumSeptember 18, 20168 min read
Zero-Bug Software Development
The Zero-Bug Policy is not a myth — it is the answer. And it can work for your team, today.Sam HatoumMay 8, 20169 min read