Swarm testing
Alex Groce, Chaoqiang Zhang, Eric N. Eide et al. · International Symposium on Software Testing and Analysis · 2012
Swarm testing is a novel and inexpensive way to improve the diversity of test cases generated during random testing. Increased diversity leads to improved coverage and fault detection. In swarm testing, the usual practice of potentially including all features in every test case is abandoned. Rather, a large “swarm” of randomly generated configurations, each of which omits some features, is used, with configurations receiving equal resources. We have identified two mechanisms by which feature omission leads to better exploration of a system’s state space. First, some features actively prevent the system from executing interesting behaviors; e.g., “pop” calls may prevent a stack data structure from executing a bug in its overflow detection logic. Second, even when there is no active suppression of behaviors, test features compete for space in each test, limiting the depth to which logic driven by features can be explored. Experimental results show that swarm testing increases coverage and can improve fault detection dramatically; for example, in a week of testing it found 42% more distinct ways to crash a collection of C compilers than did the heavily hand-tuned default configuration of a random tester.
Cited by 101 works. doi:10.1145/2338965.2336763
Authors
References
- DART
- DART
- The category-partition method for specifying and generating functional tests
- Partition testing does not inspire confidence (program testing)
- A Tool for Checking ANSI-C Programs
- Simplifying and isolating failure-inducing input
- Encyclopedia of software engineering
- Feedback-Directed Random Test Generation
- Empirical evaluation of the tarantula automatic fault-localization technique
- Introduction to Software Testing