TestEvoViz: visualizing genetically-based test coverage evolution

Abstract

Genetic algorithms are commonly employed to generate unit tests. Automatically generated unit tests are known to be an important asset to identify software defects and define oracles. However, configuring the test generation is a tedious activity for a practitioner due to the inherent difficulty to adequately tuning the generation process. Furthermore, evolution processes are most of the time compared solely using the final results, while discarding all the details of the iterations that are themselves important for an adequate tuning. This paper presents TestEvoViz, a visual technique to introspect genetic algorithm-based test generation processes. TestEvoViz offers the practitioners a visual support to expose the process and decisions made by the generation algorithm. We first present a number of case studies to illustrate the expressiveness of TestEvoViz. We then conducted a user study involving 22 participants including researchers, students and professional software engineers. Participants use our visual approach to analyze, compare and tune test generation algorithm executions. All participants were able to complete the tasks. Our findings show that participants focus more on the visual components that depict information about the test similarity, individuals coverage increments, and the final generation code coverage.