A Blog from QualiTest

Traditional Testing vs. Crowdsource Testing vs. Freelance Testing

Over the past few years, the technical workplace has experienced a shift in the management of software testing. This movement away from a “traditional” testing...

Over the past few years, the technical workplace has experienced a shift in the management of software testing. This movement away from a “traditional” testing model and towards crowdsource testing and freelance testing could be beneficial to the software community – but do the advantages outweigh the detriments?

While there are benefits and detriments to each testing model, devising a combination and highlighting the paramount values of each model can provide the most comprehensive practices for testing software.

Traditional Testing Services

A traditional testing model is typically comprised of a testing staff providing services from a test center or at the client’s facility. This model has the benefits of a reliable staff, with the advantage of convenient communication between departments, traceable test coverage, a more hands-on approach to management and more secure privacy assurances.
This testing model does suffer from disadvantages, primarily due to a smaller test base and cost. A testing group that is hired, trained, and managed by the same group of people will have similar backgrounds, and therefore is more likely to only find similar issues with a piece of software, while overlooking other defects. Furthermore, this model possesses less flexibility in size fluctuation, and might be difficult to ramp resources up and down in a fluid manner.

Crowdsource Testing

Crowdsource testing utilizes the technique of gathering contributions from a large group of people through the use of a cloud platform. These are not hired consultants and professionals, and may not even be from the same geographical area, but are instead part of an online community that contribute directly to the testing project.
The high gains of crowdsource testing are greatly due to the versatility and size of the test base. The number of testers allows the software to be tested at a greater speed, with more diverse findings. This model also benefits from the geographical, linguistic, cultural and hardware differences of the crowdsource group. The differences in environment for each user guarantees that larger gradations of defects are discovered and more versatile testing scenarios will be encountered.
However, some of the advantages of crowdsource testing can also work to its disadvantage. Because of the large test group and possible difficulties in communication, assurance of a complete test coverage and thorough bug identification can be impossible to guarantee. A crowdsource tester’s payment is dependent on the number of bugs that are found, which can lead to testers only focusing on systems during their first test cycles when there are more bugs to be found, and less inclined to get involved in more advance cycles. Furthermore, while a comprehensive testing process should commence from the start of the development process, a crowdsource testing model is only sufficient for testing during later phases of the process, such as beta testing and compatibility testing.

Freelance Testing

Freelance platforms, like odesk.com and elance.com, provide a “matchmaking” site between freelance workers and businesses looking to hire for contract or temporary jobs. Freelance testers are paid an agreed-upon amount for each contracting job.
These sites are valuable in that they can enable companies and employers to establish reoccurring freelance relationships, while retaining the cost effectiveness of a temporary employee. Freelance testing provides rewards by allowing employers to access a large pool of talent and find testers who possess a very specific skill set or a broader range of experience.
Freelance testing is accompanied by its own shortcomings. Additional management attention would be required during both the hiring and testing process to evaluate the freelancer’s abilities and performance. Furthermore, due to the nature of the engagement, the freelancer is not as committed to the clients like in other engagements; hence, dependability and reliability might become an issue affecting the deliverable’s quality.

All models are wrong; some models are useful

The statistician George Box said, “All models are wrong; some models are useful”. While there are benefits and detriments to each testing model, devising a combination and highlighting the paramount values of each model can provide the most comprehensive practices for testing software. For example, in some projects, QualiTest testing solution offers a combination of on-shore and off-shore testing, accompanied by crowdsource testing to provide the most thorough testing process. Through the expertise and resources of the traditionally modeled testing team, and the size and diversity of the crowdsourcing community, the testing can be completed in the most efficient and effective manner.