Demonstration of ThalamusDB: Answering Complex SQL Queries with Natural Language Predicates on Multi-Modal Data

Abstract

ThalamusDB supports SQL queries with natural language predicates on multi-modal data. Our data model extends the relational model and integrates multi-modal data, including visual, audio, and text data, as columns. Users can write SQL queries including predicates on multi-modal data, described in natural language. In this demonstration, we show how ThalamusDB enables users to query multi-modal data. Visitors can write their own SQL queries on two real-world data sets gathered from Craigslist and YouTube. ThalamusDB has a specialized optimizer that selects execution plans that minimize the overall cost of answering such queries. Query execution involves pre-trained neural models as well as a relational database as processing engines. ThalamusDB collects a limited number of labels for selected data items to translate similarity scores into binary predicate evaluation. Our demonstration enables visitors to compare optimized plans against naive plans in terms of processing latency. ThalamusDB allows users to trade query result precision for reduced processing overheads. Our demonstration interface enables visitors to change the performance objectives and observe their effects on final result precision as well as computation time and number of labeling requests. Similar to online aggregation, our interactive interface allows users to track shrinking error bounds during query execution.