The Challenges of Testing and the Experior Difference
Finding the right balance between testing too much and testing too little is one of the key challenges projects and businesses face when implementing new or updated systems. In some cases business stakeholders are only happy to sign off a solution by testing everything they are involved in across both the System Integration Test & User Acceptance Test phases. The impact of this is not only does it take longer to complete it’s also more costly and not an efficient way of testing. At the other end of the spectrum there are examples where organizations and projects have taken a “test light” approach leading to more defects being found in User Acceptance Testing and / or an increase in post go live incidents which ultimately impacts the overall user experience.
There are also cases where the starting point is to test a defined scope but due to project constraints and pending go live dates, testing is either cut short, the scope redefined or allowances made at the test exit meeting which wouldn’t normally be the case. The consequences of this can be significant.
It is also widely known that one of the goals of testing is to identify defects as early as possible in the testing cycle. Why? This helps to drive down the cost of testing as the sooner defects are identified the cheaper they are to fix, not only that, it also increases the speed of deployment and helps to protect your production environments from disruption. Achieving this “shift left” mentality can be difficult and therefore it is key to approach the project in the right way and be disciplined in managing the transition through the test phases.
At Experior the way we address these challenges is through our methodology. Our methodology covers six phases: Assess, Define, Prepare, Develop, Execute & Maintain. We manage the transition of each phase using quality gates and entry and exit criteria. This helps to set expectations of what is required at each phase and what should be in place before proceeding to the next stage, without this there can be a tendency for defects leaking into the following test phases, impact the ability to “shift left”.
One of the key enablers our methodology is the way we approach testing. We think it is important to test what matters most to an organization, not just based upon perception but by using fact. We do this by using our Risk Based Testing model.
How this works is firstly by defining the business events, process and scenarios. For each of these we look at level of technical risk associated taking into account the type of activity being carried out and how customized the solution is. We then look at the business impact in order to assess the frequency of usage, the financial / legal impact if it goes wrong and how long it may take to fix in the event of defect / incident. The output of our model gives a ranked and prioritized test scope which focuses on the areas that matter most and tests the true business reality. A summary of this process is shown in the diagram below.
The benefit of this approach is it reduces your test scope in a targeted and controlled manner. As shown in the diagram below, by using Risk Based Testing we see a reduction in testing effort of up to 60%. The reason being it focuses testing in the right areas and the correct phases, therefore reducing the overall cost and duration of testing. What it doesn’t do however is increase the risk or reduce the quality of the solution or outcome of the project.
If you would like to know more about our methodology, risk based testing or how we can help you target your testing in a more effective manner do not hesitate to contact us.
Originally posted June 11, 2015, by Matt Dickinson, Experior Group