RATest: Explaining Wrong Relational Queries Using Small Examples.

We present a system called RATEST, designed to help debug relational queries against reference queries and test database instances. In many applications, e.g., classroom learning and regression testing, we test the correctness of a user query Q by evaluating it over a test database instance D and comparing its result with that of evaluating a reference (correct) query Q ₀ over D. If Q(D) differs from Q ₀(D), the user knows Q is incorrect. However, D can be large (often by design), which makes debugging Q difficult. The key idea behind RATEST is to show the user a much smaller database instance D' ⊆ D, which we call a counterexample, such that Q(D') ≠ Q ₀(D'). RATEST builds on data provenance and constraint solving, and employs a suite of techniques to support, at interactive speed, complex queries involving differences and group-by aggregation. We demonstrate an application of RATEST in learning: it has been used successfully by a large undergraduate database course in a university to help students with a relational algebra assignment.