As quality leaders, and quality engineers by training, we tend to be focused on units or small bits of a problem by our very nature. While this works fine when creating or executing tests and managing defects, it doesn’t serve us well when we look at managing the overall quality of our business.
The first thing a quality leader does when taking leadership of a new team or project is to quickly assess the state of quality. This is a fantastic best practice to do immediately upon assuming the role, and yearly thereafter. However, because we are quality engineers at heart, we look at problems in very encapsulated terms. How can I improve automation? How can I improve testing in each sprint? How do I handle test data? We look at the findings and gaps as very specific, self-contained issues to be mapped, prioritized and solved. Put another way, we put the findings into a backlog, address it, put it in practice, and move on.
What we generally find by taking this approach is that while the quality in each area may be incrementally improved, there is no wholesale improvement of quality across the enterprise. Even worse, many quality leaders run into issues when challenged to show evidence of even incremental improvement by their own leadership. The difficulty many of us have as quality leaders is showing the business value of quality engineering to the overall IT and business leadership.
Through trial, error, practice and lessons learned, we’ve learned that the way to show the value of quality engineering is to focus it through the lens of business value. And this happens by regularly assessing the quality of your organization and coming up with action plans that reflect business priorities. The activities in assessing and improving each quality engineering capability aren’t the problem. It’s the analysis of the findings and creating remediation roadmaps to solve that will make or break your improvement initiatives.
Quality engineering improvement initiatives that don’t respect business priorities and show business value are doomed to fail.
How to focus on business value
Assessing your quality engineering organization so that it focuses on business value should run through three filters:
Each activity needs to align to organizational and quality governance.
Each activity needs to be measured and benchmarked against the KPIs that are right for your business.
Each activity requires collaboration between people within your quality engineering team and within your overall organization.
If any one of those three filters do not exist, then implementing them should be your priority before addressing anything such as automation, continuous testing, or test data management. Looking at the three filters in particular:
Governance are the rules, policies, and standards unique to your organization that respects its culture, abides by government regulation, and general operating procedures that create the best way to work in your organization. Quality engineering is about obtaining buy-in from internal and external stakeholders, and then using that to create a culture of quality through all parts of your organization. Quality engineering is also about following rules. Aligning any quality engineering initiatives resulting in better standards, processes, or policies must align to organizational governance. This facilitates organizational adoption.
KPIs present evidence that activities and capabilities in your quality engineering group are working as designed and evidence improvements through data. This is perhaps the best way to show that quality engineering initiatives are having demonstrable improvement. They also give you early warning that something needs to be course corrected. There are three basic rules when adopting KPIs:
Remember all KPIs don’t have to be adopted at once but should measure improvements and initiatives. The most common are things such as defect detection rate and test case execution rate. However, look for KPIs that reflect business value like uptime, defects leaked to production and user stories per sprint.
Each KPI needs to be benchmarked. Presenting metrics is a good start, but they need to be compared to something. This is either staying within a set variance, not falling below a floor value or meeting a goal over time.
Get people who consume your KPIs comparing like values. This is important, especially in massive IT organizations with a significant number of stakeholders. People can derive different insights from data, but they should all be looking at the same data with the same definition.
Finally, the most important is collaboration. Governance and KPIs allow you to align quality engineering initiatives to corporate best practices and measure their impact. However, a collaborative culture makes sure that the correct initiative you choose is the right initiative. For example, one of the first insights quality leaders gain when assessing their organization is that automated testing should be faster, cover more functionality, and run independently at scale. The general solution is to use frameworks built upon Selenium or something similar. This is the correct solution, and one that is widely adopted. However, this also depends upon having technically trained quality assurance consultants or SDETs (Software Development Engineers in Test). What if your organization hired business analysts with domain knowledge but little technical knowledge? Is adopting a development-heavy framework the right test automation initiative? Or should you look for automation tools that are more friendly to non-technical testers? Poorly chosen initiatives are usually the result of not collaborating with people around you.
Once governance, KPIs, and a collaborative culture are in place, then every gap or opportunity identified through your assessment can be solved with the initiative that is correct, and more importantly, right for your company. This will allow quality engineering to be ingrained into the business so that the quality-first mindset it creates drives business value.
