RASAT

Introduction

RASAT^[1] is a state of the art model in multi-turn Text2SQL. It supports both single-turn and multi-turn modes.

It modifies the encoder module of T5^[2] by replacing the Multi-Head Self Attention layer with a custom layer named Multi-Head Relation-Aware Self-Attention

It can be considered as a form of GNNs^[3].

Graph Construction

Incoming Technicalities

The reader may need a basic knowledge of Graph Theory concepts to understand the construction process.

From the questions^[4] and the database schema A graph will be constructed in a way that it encodes the relation between the columns, tables and tokens.

Formally

The input graph is constructed as follows:

represents all the tokens in the database schema and all asked questions
is the relation between vertices that will be detailed in the subsequent sections.

Schema Encoding

Schema encoding relations refer to the relation between schema items, i.e.,
. These relations describe the structure information in a database schema.

In RASAT, the supported types of Schema encodings are:

denotes that column is a primary key of table
denotes that column is a column of table
denotes that column is a foreign key referencing column

Schema Linking

Schema linking relations refer to the relations between schema and question items, i.e., or vice versa.

In RASAT, N-gram matches is used to indicate question mentions of the schema items. Both and are supported.

Question Dependency Structure

This relation defines a relation between different question tokens and in the same question

This relation reflects the grammatical structure of the question.

Co-reference Between Questions

This type of relation is unique to the multi-turn scenario. In a dialog with multiple turns, it is important for the model to figure out the referent of the pronouns correctly.

It relates and

Database Content Mentions

Instead of mentioning the table or column names, the user could mention the values in a specific column. In this case, the informative mention could escape from the aforementioned schema linking.

For and two modes are supported:

Notes & References

Jiexing Qi, Jingyao Tang, Ziwei He, Xiangpeng Wan, Yu Cheng, Chenghu Zhou, Xinbing Wang, Quanshi Zhang, & Zhouhan Lin. (2022). RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL.↩︎
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, & Peter J. Liu. (2023). Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.↩︎
Graph Neural Networks.↩︎
For single turn agents, ↩︎