A mixture of “crowd” and “outsourcing,” Merriam-Webster defines crowdsourcing as “the practice of obtaining needed services, ideas, or content by soliciting contributions from a large group of people and especially from the online community rather than from traditional employees or suppliers.” In crowdsource testing, this term applies to the action of using the educated masses to contribute to your testing process. This process had clear advantages, such as reaching a wider range of testers and a potentially higher ROI for the testing process. However, there are certainly disadvantages as well, such as difficulties in confidentiality and communication between all parties involved.
- Cost-effective: the company only pays for bugs which are found instead of an hourly or salaried rate which professional testers would receive
- Vast range of users provide huge diversity in their experiences
- Allows for testing with all kinds of different parameters, such as with different connection speeds, browsers, and devices to which the core testing team may not have access
- Larger group is more likely to find reproducible bugs than a handful of testers
- Lack of bias towards the company can be expected of testers
- Monetary value of both of the above= more thorough testing for an equal price range in a shorter amount of time sans contract and overhead
- Confidentiality is compromised by having testing performed by a large group who may or may not have much loyalty to the brand/product
- Communication between testers can be difficult due to time or language barriers
- Test coverage can be more difficult to guarantee and can therefore require more managerial oversight to ensure thorough bug identification
- Because payment usually depends on the number of bugs found and not their severity, testers may seek out and identify many small, less-important bugs rather than devote a lot of time and effort to finding one or two large, debilitating bugs
- This, combined with the exploratory nature of crowdsourcing, means that it can be difficult to ensure that the entire product has been tested thoroughly enough to ensure usability
Some companies prefer to use crowdsourcing in addition to or support of their own in-house testing teams. This provides a much more in-depth testing process, but can sometimes lead professional testers to feel that their skills are being undervalued and outsourced to those who are less qualified. If a crowdsource tester finds no bugs, they are not paid; in the eyes of a professional tester, that can be seen as relating their hard-earned skills and education to unpaid labor. It can also undermine much of the cost-effective nature of crowdsourcing, because the company has to pay their test teams’ wages on top of the per-bug compensation.
In the end, crowdsourcing software testing often proves to be beneficial to developers as well as testing companies. However, like any testing methodology, it is not going to be the right decision for everyone; for more information, please visit the Crowdsourcing section of our website here.
Written with thanks to Andy Dowson