The client is a large telco operator that has the UK’s biggest mobile network. On average, it provides the fastest 4G speeds available in the country, and covers 97% of the UK population with this network.
Business Needs and Objectives
The client required a method for testing the stability of their different mobile network environments, both development and stable environments. The previous set-up required having employees arrive at the office early to conduct a small number of manual tests covering some aspects of 3 environments. Results would then be delivered verbally during a conference call along with any new or updated defects in an environment.
The client chose QualiTest as we were already a reliable consultant of the company and able to provide a solution they could not.
The expected outcomes were:
- Execution of tests outside normal business hours
- Detailed daily report made available by the start of the working day
- Improved test coverage compared to current manual testing
- Tracking of defects raised/actioned
- Highlighting of time periods with consistent problems
- Reusable and flexible test environment
The QualiTest Solution:
QualiTest’s understanding of the market and client’s needs convinced us to recommend an automation based solution. Automation would permit a flexible list of test cases to be executed on an open schedule while allowing a much wider range of test cases to be covered without the constraint of manual input. The automation tool would have to interact with a wide variety of user interfaces as well as being able to compose detailed reports on findings or possible defects. The final solution would also need to be as versatile as possible, as well as being easy to use.
This solution was to be paired with off-shore resources who would more deeply investigate reported failures from the automation tool and raise defects or change requests where needed. Reports would be automatically generated for the off-shore resources to add to before being delivered to the client at the start of their working day. The global reach of QualiTest enabled a cost and time effective solution to the off-shore resources by utilizing QualiTest India’s expertise and professionalism. Dedicated members of staff available at a normal working hour for their region are able to execute retests and construct a detailed report before normal working hours in the UK begin. The only issue with this would be the requirement for the tests to be executed locally.
QualiTest’s unique insight into tools that would provide a solution to these requirements led us to recommend TestPlant. TestPlant provide both a versatile and reliable cross platform automation test tool, eggPlant, as well as an expertly constructed remote connection cloud solution, eggCloud. Using eggPlant, two automation engineers were able to cover a wide variety of test cases while also allowing the end user to specify variations in the test with relative ease. QualiTest’s automation specialism enabled the solution to be extremely versatile as testing could be conducted on a variety of sim cards, radio frequency signals, handsets, and test cases. While the new solution covered over 440,000 possible combinations, the eggCloud tool was required for in-depth investigation into failures as well as a solution for one-off tests that would not be cost or time effective to automate. eggCloud enabled the off-shore resources to directly interact with devices and execute tests as if they were on site with the client. The user interface displayed on each device would be reflected accurately through the cloud solution.
- Reusable test code
- Reliable testing
- Testing is able to be executed at any time and for any period of time
- Report is more reliable and automatically generated
- Better SIM card tracking ensured more accurate results
- More in-depth report able to be generated by offshore resources
- Test coverage is improved
- Free up time for on-site employees
- Defect tracking is clearer
- Defects are more closely monitored
- Health checks now executed 3 times a day
- Covers over 442,000 possible test case combinations
- Projects covered by off-shore are now more cost effective
- Customer satisfaction is encouraging the solution be used more widely throughout the company
- Provide an optional solution to new avenues of testing