A Hybrid Algorithm for Multi-objective Test Case Selection

Abstract

Testing is crucial to ensure the quality of software systems – but testing is an expensive process, so test managers try to minimise the set of tests to run to save computing resources and speed up the testing process and analysis. One problem is that there are different perspectives on what is a good test and it is usually not possible to compare these dimensions. This is a perfect example of a multi-objective optimisation problem, which is hard — especially given the scale of the search space here. In this paper, we propose a novel hybrid algorithm to address this problem. Our method is composed of three steps: a greedy algorithm to find quickly some good solutions, a genetic algorithm to increase the search space covered and a local search algorithm to refine the solutions. We demonstrate through a large scale empirical evaluation that our method is more reliable (better whatever the time budget) and more robust (better whatever the number of dimensions considered) – in the scenario with 4 objectives and a default execution time, we are 178% better in hypervolume on average than the state-of-the-art algorithms.