Topological Relation Aware Transformer

Abstract

We present a Topological Relation Aware Transformer (T-RAT), a specialized head transformer to open sets, an element of the topology τ generated by the set S, the set of all pre-existing relations between input tokens of the model. From this topological space (S, τ), we present the way to spread each open set to one head of our Transformer. T-RAT improves exact match accuracy in Text-To-SQL challenge (62.09%) without any enhancement of large language models compared to the baseline models RAT-SQL (57.2%) and Light RAT-SQL (60.25%). Keywords: Deep learning, Natural Language Processing, Neural Semantic Parsing, Relation Aware Transformer, RAT-SQL, Text-To-SQL Transformer.