There's an art to reporting QA metrics to clients. It's a people skill. View it as an opportunity to build consensus, not a way to appear clever, confuse clients, or disguise issues.
Are you developing and testing software for a client? Maybe you're partnering with them as you build a Minimum Viable Product (MVP) intended for wider distribution. If so, you're in luck. Today, we're looking at how you can decide which QA metrics to surface to your clients, with some suggestions about the numbers they'll find most relevant.
How to decide what matters to clients
There are a few of basic questions to start with:
- Does this metric impact core client concerns like budget, timelines or requested features?
- Can one metric represent general trends in place of multiple metrics?
- Is there anything you definitely need to include, or anything that is lower priority?
Once you've done some initial triaging with these questions in mind, look at the QA metrics you have left.
First of all, don't base your decisions on emotions or ego. Just because your velocity metric shows that there are, err... opportunities to improve testing or development efficiency, that doesn't mean you should hide it. In fact, it might mean you need to have a conversation with the client about more resourcing or perhaps including some test automation. Give them the context they need to make decisions. For example:
'This epic has 25 issues/ tickets with a tolerance of five and will probably need five to ten bug fixes. Our velocity is 40 issues per sprint, so we anticipate we can complete this epic in one sprint, including in-house and third-party testing. Our sprints last two weeks.'
Next up: include anything around speed or performance metrics and the testing processes that may be less obvious (but that absolutely must happen). Often, these foundational elements are necessary to get right before moving on to client-side feature requests.
Finally, try not to include anything that requires detailed technical knowledge to understand. Avoid jargon and acronyms. We're not saying be vague, just look for what you can report on in a clear and concise way. When in doubt, give context:
'We found opportunities to improve application speed by testing top wait times for retrieving data from memory, which currently stand at x.xx seconds.'
Which QA metrics to surface
Here are some QA project management metrics we suggest bringing to the table:
- Your position on the top-level roadmap, and recent or upcoming testing milestones.
- If you run an Agile testing process, your progress at a story or epic level (not tasks!).
- Cumulative flow, showing testing velocity, backlog and current activity.
- Number of bugs found, in progress and solved, highlighting blockers and high severity issues.
- Some detail in your client's priority areas, such as performance, security, or adaptability.
A final note on how to "speak client"
When communicating updates to clients, follow three key tenets:
- Check your math. Check it again.
- Have a relationship manager run the meeting.
- Always provide context for metrics, ideally with a visual aid.
You'll notice we've assumed you will be communicating these QA metrics in a meeting. Do not be tempted to send some kind of automated data aggregation in a weekly email (looking at you, developers and managers).
The point is that you've tailored your QA metrics so they're precisely what's important to the client. So, relay that information with the care and attention it deserves. Your client will thank you.
Test plan vs test case (and why you need both)
Poor software doesn’t provide value to users. It causes customers to complain, cancel...
Getting to the heart of a great test plan
Think of test planning like raising children. Let them loose and you’ll soon find...