Ask your data in English, get governed SQL or an honest no

Ask a knowledgebase built from five annual water-quality PDFs "What was the average turbidity by monitoring site in 2024?" and it returns three rows: Eastern Catchment Intake 2.4, Northern Reservoir 2.17, Southern Treatment Plant 2.51. The values match the corpus ground truth to ±0.01.

A fourth site, Western Bore, is correctly absent. The FY2024 report carried no readings for it, and the answer does not make up an average to fill the gap.

Whether a compliance lead can use a result like this depends on the code between the model and the database, and on what the screen shows when there is no answer.

One planning call, then a guard

The resolver makes one LLM call per question, at temperature 0. First it introspects the knowledgebase's live DuckDB catalogue and puts three things in the prompt: every ontology.* view with its column types and stored COMMENT semantics, the LinkType list with its pre-validated join conditions, and a corpus identity line. The model returns JSON with three fields: sql, object_types and a one-sentence reason.

A deterministic function, validate_sql in strata/resolver.py, then decides whether the SQL reaches the database. It rejects empty SQL, SQL that reads raw_extracted directly, SQL that names no governed view, any view name the catalogue does not know, and more than three views in one query (MAX_JOINS = 3, the KINETIC-FUNC-002 constraint).

raw_extracted is where the tables scraped from the PDFs live. A governed answer never reads them directly, only the unified views built on top, which carry the canonical column names, the TRY_CAST discipline and the _source_document provenance columns. Whatever passes the guard is wrapped in SELECT * FROM (…) LIMIT 1000 before it runs. The rejections are pinned by unit tests:

assert "ontology.*" in validate_sql("SELECT * FROM raw_extracted.t", known)
assert "unknown view" in validate_sql("SELECT * FROM ontology.nope", known)

The PRD's §10.1 sketch had the resolver doing six steps: matching ObjectTypes by embedding similarity over their descriptions (with an LLM tie-break when ambiguous), property matching, a join-path search, view resolution, SQL generation, and classifying the kinetic type. What shipped folds the matching into the one grounded call and moves enforcement into a pure function with no I/O, which is what lets the unit tests above pin it. The resolver work closed with 106 unit checks and 23 live checks; its live proof was a generated join across a LinkType path returning a cross-document average of 3.65 for Eastern Intake (3.5 and 3.8, from two different PDFs), with both documents in the provenance list.

What comes back with the number

Under the result table sit five provenance chips, one per source PDF, the views used, and a View governed SQL disclosure:

SELECT site, ROUND(AVG(turbidity_ntu), 2) AS avg_turbidity_ntu
FROM ontology.water_quality_observation
WHERE EXTRACT(YEAR FROM record_month) = 2024
GROUP BY site
ORDER BY site

For aggregate answers, provenance is the distinct set of source documents behind the views read. For row-level answers the prompt tells the model to also select _source_document and _source_table on every row. Those columns are hidden from the rendered table, and they let each row walk back to the exact table region in the exact PDF it came from.

Rule 4 of the prompt rounds averages to two decimals, which is where the 2.17 in the screenshot comes from.

The refusal card

Ask the same knowledgebase "What is our employee churn rate?" and you get a card reading No matching data model, with the resolver's own sentence underneath: "The data model contains no employee or staff information, so employee churn rate cannot be calculated."

Under rule 5 of the prompt, a model that cannot answer from the listed views returns empty SQL and explains in reason; if the guard rejects the model's SQL instead, both reasons are joined onto the card. Either way the user sees the same card. I think that is right, because the user needs to see plainly that nothing ran.

A water-quality corpus has no employees, but a system allowed to divide one plausibly named column by another will still produce a churn figure.

Three verdicts in the monitors

Monitors run unattended on a Celery beat tick, so the same discipline matters more there. A dynamic-threshold monitor compares a property against another table instead of a constant, here an enterprise agreement pay floor: hourly_rate < governed_by.min_hourly_rate, where governed_by is a stored LinkType and min_hourly_rate is a number read out of a scanned award schedule. The config carries no SQL. The compiler turns it into a correlated EXISTS, so exactly one row survives per local instance. Page 146: the number nobody was going to find walks through that monitor and its compiled SQL.

The compiler also returns a second query, the no_data complement. A row is breach only when both operands cast to a finite number and the comparison holds. It is no_data when its own operand is NULL or un-castable, or when no castable floor row exists to compare against. Everything else is compliant. The evaluator records the no_data count with a capped sample and an operand-side reason (left_null, right_null or right_missing), and the alert row carries a no_data badge, so a "0 breaches / N no-data" evaluation does not read as a pass.

The Aruma demo corpus's payroll export has 57 employees on real classification and pay-point codes, four of them paid below their 1 July 2025 floor. Employee E1057 is on classification "DDSO 7", which does not exist in the agreement; it runs DDSO 1 to DDSO 6. A two-way monitor finds no floor for E1057, the comparison is not true, and the row counts as compliant. With three verdicts it is right_missing, counted and sampled for somebody to review.

The second query is there because decision D14 in the monitor design requires that "a missing award floor can never masquerade as compliant".

A false refusal on the Aruma knowledgebase

A knowledgebase named Aruma held that organisation's enterprise agreement. Someone asked the AI FDE (forward deployed engineer) which Aruma staff were paid below the floor, and the resolver refused because no column identified staff as belonging to Aruma. Every row already belonged to Aruma; the model had read the corpus title as a data filter. Four employees below their legal minimum covers that run.

The fix is a prompt section, _kb_identity_section, which reads the knowledgebase name and description from metadata and states that corpus-title words are not data filters, and that <title-word> staff means all rows. It is domain-agnostic and returns an empty string on any failure, so it cannot break a resolve. Three deterministic probes that had false-negatived on a division filter then passed 3 of 3.

We had already grounded the FDE's own prompt against this, but the FDE is one consumer of the resolver; the Ask tab, the monitors, the dashboard seeder and the agent nodes still hit the raw resolver prompt. Every consumer needed the grounding, so it went into the resolver prompt itself.

The same resolver behind the agents

Besides the Ask tab, the resolver is a library used by the monitors and the dashboard seeder, and an HTTP endpoint pair (.../ontology/resolve and .../ontology/query) that the Agent Builder's ontologyQuery node calls over the standard service boundary. The older text2sql node makes the agent author wire up a schema source, then trusts generated SQL over raw tables; ontologyQuery hands the problem to the resolver and gets back canonical names, pre-validated join paths and provenance.

The agent answers eight plain-English questions over the Ask tab's governed model, from an agreement whose Schedule J is 41 pay rows across 4 effective dates and 3 measures, 492 numbers in all. Each of the eight queries goes through the same guard as the Ask tab, so any of them could have come back as a refusal card.

What I have not measured

I have no per-query latency figure, and no labelled evaluation set for refusals, so I cannot give a false-refusal rate. My refusal-quality evidence is the 3-of-3 probe set from the identity fix and the demo seed's --verify mode. A refusal benchmark, pairing questions the model should answer with ones it should not, does not exist yet.

A known synthesis convergence bug remains: with many same-shaped tables, a round can temporarily leave a backing table out of a view, and the next round usually absorbs it. When that happens the resolver still answers, over less data than it should have used. A deterministic grouping fix is registered; it is not fixed today.

The demo environment rebuilds with python3 scripts/strata_demo_seed.py --recreate (about 30 minutes), and --verify re-asserts the natural-language answers against ground truth, including the three turbidity values and the absent Western Bore.