How it Works · Grounded SQL

the pipeline

Nothing gets executed on faith

The schema is narrowed before generation, and the SQL is proven correct by running it, never by how it looks.

01

Ask

A question in plain English, plus the target database, typed in, or a schema/.sqlite file you provide yourself.

app.py

Formatted as a Mistral instruction:
[INST] {system_prompt}
Database Schema: {schema}
Question: {question} [/INST]

→

02

Hybrid retrieval

BM25 (exact column names) and FAISS (semantic meaning) each rank the schema's tables, fused so only relevant tables reach the model.

retrieval/hybrid_linker.py

Reciprocal Rank Fusion, k=60:
score(t) = Σ 1 / (60 + rank_i(t))
summed across the BM25 and FAISS rankings.

→

03

Fine-tuned generation

Mistral-7B-Instruct-v0.2 + a QLoRA adapter writes the SQL, served INT8-quantized and cached in Redis by question+schema hash.

serving/model_loader.py

r: 64 alpha: 128 dropout: 0.05
target: q/k/v/o_proj, gate/up/down_proj

→

04

Execution check

The SQL is actually run against SQLite. Correctness is measured by comparing the returned rows, never by how the query looks.

evaluation/executor.py

Rows compared as order-insensitive sets: NULLs normalized, floats rounded to 4dp, strings lowercased.

why it's built this way

Five decisions that shaped the system

Each one traded simplicity for a measurable gain, not for its own sake.

01

Execution accuracy over string match

Two SQL queries can look completely different and return identical rows. Only running them tells you if a query is actually correct.

02

BM25 alongside semantic search

Dense retrieval finds "revenue" → amount, but misses exact identifiers. BM25 finds customer_id verbatim. Fusing both is what eliminated hallucinated columns entirely.

03

INT8 over 4-bit for serving

SQL correctness is binary, not a matter of degree. A 3.1% accuracy drop from more aggressive quantization means roughly 32 more wrong queries on the 1,034-query test set, too costly for a two-and-a-half times throughput gain.

04

Loss masked to SQL tokens only

Training loss is computed only on the generated SQL, not on the schema or question prefix, so the model learns to write SQL, not to predict the schema back.

05

Tanglish as a documented failure, not a hidden one

The model breaks on Tamil-English code-switched questions, a schema-linking failure on Romanized words. Naming that precisely is more useful than pretending it doesn't happen.

results

Why fine-tuning wins on the 1,034-sample test set

Four systems, scored the same way: execute the generated SQL, compare its rows to the gold query's rows.

Non-obvious finding: on the 166 held-out schemas, adding retrieval to the fine-tuned model hurt accuracy (70.2% → 66.4%). Cross-schema retrieval quality, not model capacity, is the real ceiling on generalization.