String Matching

Quick-start

It is the most naive comparison function. it calculates the fraction of predictions that are exactly identical with the ground truth query.

Notations

The String matching accuracy of between and is defined as follows:

This metric is too restrictive, as it penalises on any difference, even if the prediction is still equivalent to the ground truth query.

Example

Let:

the database defined by:

CREATE TABLE PERSON(
ID int primary key,
FIRST_NAME VARCHAR(20),
LAST_NAME VARCHAR(20)
)

SELECT FIRST_NAME,LAST_NAME from PERSON where FIRST_NAME==LAST_NAME;

Suppose that the predicted query is:

 SELECT FIRST_NAME,LAST_NAME from PERSON where LAST_NAME==FIRST_NAME;

Then, even both queries are semantically equivalent, the prediction is considered incorrect with respect to