SBST Tool Competition 2021

Sebastiano Panichella, Alessio Gambi, Fiorella Zampetti, Vincenzo Riccio

Publikation: Beitrag in Buch/Bericht/KonferenzbandKonferenzbeitragBegutachtung


We report on the organization, challenges, and results of the ninth edition of the Java Unit Testing Competition as well as the first edition of the Cyber-Physical Systems Testing Tool Competition. Java Unit Testing Competition. This year, five tools, Randoop, UtBot, Kex, Evosuite, and EvosuiteDSE, were executed on a benchmark with (i) new classes under test, selected from three open-source software projects, and (ii) the set of classes from three projects considered in the eighth edition. We relied on an improved Docker infrastructure to execute the tools and the subsequent coverage and mutation analysis. Given the high number of participants, we considered only two time budgets for test case generation: Thirty seconds and two minutes. Cyber-Physical Systems Testing Tool Competition. Five tools, Deeper, Frenetic, GABExplore, GAB Exploit, and Swat, competed on testing self-driving car software by generating simulation-based tests using our new testing infrastructure. We considered two experimental settings to study test generators' transitory and asymptotic behaviors and evaluated the tools' test generation effectiveness and the exposed failures' diversity. This paper describes our methodology, the statistical analysis of the results together with the contestant tools, and the challenges faced while running the competition experiments.

TitelProceedings - 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing, SBST 2021
Herausgeber (Verlag)IEEE
ISBN (elektronisch)9781665445719
PublikationsstatusVeröffentlicht - 31 Mai 2021
Extern publiziertJa


NameProceedings - 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing, SBST 2021


Untersuchen Sie die Forschungsthemen von „SBST Tool Competition 2021“. Zusammen bilden sie einen einzigartigen Fingerprint.

Dieses zitieren