Test data generation for program units using a game with a purpose

One of the most important, but tedious and costly tasks of the software testing process is test data generation. The challenge is finding approaches in which humans could generate test data through more attractive, faster, and cheaper ways. One approach is using Game with A Purpose in the process of test data generation. In our previous work, we introduced two games called Rings and Greenify, by which many inexpensive players, with no special technical abilities, become engaged in test data generation. Despite the promising results of Rings and Greenify, they have certain limitations and issues. In this paper, we present a new GWAP for test data generation, called QOTE, in order to improve the application of GWAP in test data generation for program units. QOTE provides a different game-play and has certain advantages compared to prior gamesc. Experimental results have shown that QOTE outperforms prior games from two aspects: game quality and capability of test data generation. We have conducted another experiment based on mutation analysis to further evaluate test data generation capabilities of QOTE compared to four automatic approaches and show that the test data generated by QOTE can reveal more failures compared to the mentioned automatic approaches.