Tiddle: a trace description language for generating concurrent benchmarks to test dynamic analyses

Dynamic analysis is a promising technique for finding concurrency bugs in multithreaded programs. However, testing a dynamic analysis tool can be difficult. Researchers end up writing large amounts of small benchmark programs. Since the benchmarks themselves are concurrent programs, they may execute nondeterministically, complicating testing of the analysis tool. We propose testing dynamic analyses by writing traces in a simple trace description language, Tiddle. Our implementation, written in Haskell, generates deterministic multithreaded Java programs for testing dynamic analyses. We report that it is substantially easier to write programs with incriminating bugs such as race conditions in Tiddle than the corresponding Java source code version, reducing the amount of source code to maintain and understand. Although our implementation is targeted towards Java, the ideas extend to any other languages which support mutable fields and multiple threads.