% Querying with AI Linguistic Trees:
% Stanza-Backed Dependency Parses, Lemma Normalisation, and DuckDB
% for Sub-Millisecond Query over Large Historical Document Sets
%
% ID: ai-stanza-linguistic-trees
% Status: draft v0.1 (2026-05-04)
% Companion package: agents/data-ml/ai-stanza
% Related papers:
%   - ai-scholar-document-infrastructure (PDAI / structure.xml)
%   - ai-index-retrieval-system           (multi-layer index, OED-style concepts)
%   - pdai-page-centric-document-format

\documentclass[11pt]{article}
\usepackage[a4paper,margin=2.5cm]{geometry}
\usepackage{hyperref}
\usepackage{amsmath}
\usepackage{booktabs}
\usepackage{listings}
\usepackage{xcolor}
\usepackage{microtype}
\usepackage{graphicx}

\hypersetup{
  colorlinks=true,
  linkcolor=blue!50!black,
  citecolor=blue!50!black,
  urlcolor=blue!50!black
}

\lstdefinelanguage{sqltight}{
  language=SQL,
  basicstyle=\ttfamily\small,
  keywordstyle=\color{blue!60!black}\bfseries,
  commentstyle=\color{gray}\itshape,
  stringstyle=\color{red!60!black},
  showstringspaces=false,
  breaklines=true,
  frame=single,
  framesep=4pt,
  framerule=0pt,
  backgroundcolor=\color{gray!7},
  numbers=none
}

\title{Querying with AI Linguistic Trees\\[2pt]
\large Stanza-backed dependency parses, lemma normalisation, and DuckDB
for sub-millisecond query over large historical document sets}

\author{Gerd Graßhoff\\
  Humboldt-Universität zu Berlin\\
  \texttt{gerd.grasshoff@hu-berlin.de}}

\date{\today \quad — \quad Draft v0.1}

\begin{document}
\maketitle

\begin{abstract}
We present \texttt{ai-stanza}, a linguistic-index layer for large historical
document collections that combines (i)~per-unit language detection and
multi-language Stanford Stanza pipelines, (ii)~on-disk \emph{stanza
documents} (CoNLL-U + serialised \texttt{stanza.Document} JSON) co-located
with each source document, and (iii)~a denormalised DuckDB schema designed
so that the four query shapes that dominate scholarly use of historical
corpora --- lemma+POS lookup, predicate-argument (subject--verb--object)
search, named-entity co-occurrence, and provenance lookup --- run in
sub-millisecond time on tens of millions of tokens. \texttt{ai-stanza}
sits as an autonomous add-on on top of the page-centric PDAI infrastructure
\cite{pdai-page-centric}, consuming logical units (sections, letters,
catalog entries, fallback pages) produced by \texttt{ai-scholar}
\cite{ai-scholar-arxiv} and feeding the cross-corpus concept layer of
\texttt{ai-index} \cite{ai-index-arxiv}. We argue that lemma normalisation
plus dependency parses, stored in a columnar embedded database with a
small number of intentionally denormalised columns, gives historians
\emph{search without compromise}: deterministic, reproducible, fast,
multi-language, and structurally aware, on a single laptop SSD.
\end{abstract}

\tableofcontents
\bigskip

\section{Introduction}
\label{sec:intro}

Two failure modes dominate the digital study of large historical corpora.
The first is \emph{full-text search}: the historian types the surface form
of a word, the engine returns documents that contain it, and the historian
discovers --- after weeks of work --- that the verb she wanted (Latin
\emph{attrahere}) appeared in twenty-eight inflected forms, of which she
caught nine. The second is \emph{vector retrieval}: the system returns
``similar'' passages, but cannot answer the question that actually mattered
--- \emph{is X the subject of Y in this sentence?} --- because that is a
question about grammar, not about embedding distance.

This paper describes a small, opinionated piece of infrastructure that
addresses both failure modes by going one level deeper into language. We
parse every textual unit of every PDAI document with a per-language
Stanford Stanza pipeline \cite{stanza-paper}, store the resulting
dependency trees and lemmas in a columnar embedded database
\cite{duckdb-paper}, and expose a query layer aimed at four
question shapes that, in our experience, account for the overwhelming
majority of scholarly queries on historical text. We call the result
\emph{querying with AI linguistic trees}: trees because the unit of
analysis is the dependency parse; AI because the parse is produced by a
neural model trained on Universal Dependencies treebanks; linguistic
because the queries are normalised at the level of lemma and grammatical
relation, not surface string.

The package, \texttt{ai-stanza}, is an add-on to an already-deployed
document infrastructure, \texttt{ai-scholar}, which produces page-level
OCR XMLs, a semantic \texttt{structure.xml} layer, and a per-document
DuckDB index. Treating that infrastructure as read-only input,
\texttt{ai-stanza} adds linguistic projections in a strictly additive
way (Section~\ref{sec:integration}) and a global aggregate database for
cross-corpus queries.

\paragraph{Contributions.}
\begin{enumerate}
  \item A four-kind \emph{Unit} abstraction (\textsf{section}, \textsf{letter},
    \textsf{catalog}, \textsf{page}) that lets a single linguistic engine
    serve very different scholarly genres.
  \item A \emph{language-first} pipeline rule: no Stanza model is loaded
    before the per-unit language is confirmed, eliminating a class of
    silent miss-lemmatisation bugs that disfigure many existing pipelines.
  \item A \emph{denormalised dependency} schema that turns the
    ``who-does-what-to-whom'' query --- expensive in canonical normalised
    form --- into a single indexed scan, making it interactive on
    $10^7$~tokens.
  \item An \emph{on-disk stanza document} convention (CoNLL-U + JSON, one
    file per logical unit) co-located with the source PDAI document, so
    downstream tools (treebank exporters, linguistic editors, training
    data builders) get a clean handoff with no re-parsing.
  \item A reproducible specimen suite spanning four genres
    (Section~\ref{sec:examples}) --- 18th-century scientific Latin,
    Renaissance Latin/German correspondence, mathematical-historical
    epistolary text, and 20th-century technical German/English --- to
    show that the same engine carries.
\end{enumerate}

\section{Related Work and Motivation}
\label{sec:related}

\paragraph{Stanford Stanza.}
Stanza~\cite{stanza-paper} provides neural pipelines for tokenisation,
sentence splitting, multi-word token expansion, POS tagging, lemmatisation,
dependency parsing, and named-entity recognition for over 70 human
languages. It is the de-facto open-source successor to
StanfordCoreNLP~\cite{corenlp-paper} for non-English work, and crucially
includes high-quality models for classical Latin
(\texttt{proiel}~\cite{proiel-treebank}, \texttt{ittb},
\texttt{udante}) and Ancient Greek (\texttt{proiel}, \texttt{perseus}),
which are essential for our corpora. Recent reviews place Stanza on or
near the Pareto front for accuracy and ease-of-deployment in
research-grade pipelines \cite{ud-survey}.

\paragraph{Universal Dependencies.}
Our query layer takes the labels of the Universal Dependencies (UD) project
\cite{ud-paper, ud-v2} as primitives: \texttt{nsubj},
\texttt{obj}, \texttt{nmod}, \texttt{amod}, \texttt{advmod}, \texttt{root},
and so on. UD is the lingua franca of cross-treebank work; phrasing
``find sentences where Y has nsubj X'' yields a query that can be
re-targeted to any UD-parsed corpus.

\paragraph{DuckDB.}
DuckDB \cite{duckdb-paper, duckdb-tpch} is a columnar, vectorised,
single-binary OLAP database. For our load --- read-mostly, group-by-heavy,
join-light, gigabytes-not-petabytes --- it occupies a sweet spot that
classical row-stores (PostgreSQL) and Lucene/Solr-style inverted indexes
do not. It also embeds in Python, ships with the package, and exposes
the same SQL one writes against PostgreSQL, which matters for
reproducibility.

\paragraph{The PDAI infrastructure.}
\texttt{ai-stanza} builds on the PDAI (Portable Document AI) format
\cite{pdai-page-centric, ai-scholar-arxiv}, which records each document
as a directory of page-centric XML files plus a semantic
\texttt{structure.xml} layer organised as a flat list of typed sections
(\texttt{caput}, \texttt{propositio}, \texttt{demonstratio}, $\ldots$).
That format is what makes \emph{section-as-unit} parsing tractable: the
text already comes pre-segmented at the level of authorial argument, with
inline page-break and formula placeholders.

\paragraph{What is genuinely new here.}
The components --- Stanza, UD parses, DuckDB, CoNLL-U --- are well-known.
What is new, in our view, is the \emph{combination tuned for one specific
question class}: ``what does this corpus say, in lemma- and grammar-aware
terms, about $X$?'', where $X$ is a person, a concept, an action, or a
predicate-argument structure. We make that question fast, deterministic,
and reproducible at corpus scale by adopting four design decisions
(Section~\ref{sec:design}) that follow from a single goal: \emph{queries
that historians want should not be the slow path}.

\section{Design Rationale}
\label{sec:design}

\subsection{Four query shapes, four indices}

Empirical inspection of how scholars actually query our corpora reveals
that the questions cluster into four shapes:

\begin{enumerate}
  \item \textbf{Lemma + POS}: ``all forms of \emph{aqua} used as a noun.''
  \item \textbf{Predicate-argument (SVO and friends)}: ``find sentences
    where Y has X as nsubj.''
  \item \textbf{Named-entity co-occurrence}: ``persons co-occurring with
    places in the same sentence.''
  \item \textbf{Provenance / reconstruction}: ``give me the full Stanza
    parse of unit~$U$.''
\end{enumerate}

The four shapes drive four indices on the schema in
Section~\ref{sec:schema}. Anything outside these shapes is a join away,
but these four are the ones we make $O(\text{ms})$ at $10^7$~tokens.

\subsection{Language detection is mandatory and first}

A wrong-language Stanza pipeline silently produces wrong lemmas and wrong
dependency edges. The Latin word \emph{via} processed with a German
pipeline becomes the German preposition \emph{via}; the dep edges are
nonsense; the bug is invisible because the rows look fine and the queries
run.

We therefore enforce: \emph{no pipeline is loaded before the per-unit
language is confirmed.} Resolution proceeds in priority order through
manifest hint, \texttt{xml:lang}, Stanza's built-in
\texttt{langid}~\cite{stanza-langid}, and a \texttt{lingua}-based fallback
for short or mixed units. Below confidence threshold and without a hint,
the unit is \emph{skipped} (and reported), never silently parsed.

\subsection{Denormalised dependencies are not premature optimisation}

The dominant query, \emph{``X as nsubj of Y''}, requires in canonical
normalised form three indexed scans and two hash joins:

\begin{equation}
  \texttt{deps} \,\bowtie\, \texttt{tokens}_{\text{head}} \,\bowtie\,
  \texttt{tokens}_{\text{dep}}
\end{equation}

On $10^7$~tokens this is comfortably $200$--$500$~ms in DuckDB --- ten to
fifty times slower than a human's interactive perception threshold. We
duplicate \texttt{head\_lemma}, \texttt{head\_upos}, \texttt{dep\_lemma},
\texttt{dep\_upos} onto the dep row itself, paying $\approx 30$~bytes per
token for two orders of magnitude on query latency. For a workload that is
overwhelmingly read-only and overwhelmingly composed of these queries,
this is the right trade.

\subsection{An autonomous package, not a function}

The Stanza-on-PDAI work was prototyped as a function inside
\texttt{ai-scholar} (Latin only, sections only, hard-coded to the Euler
pilot's placeholders). Promoting it to a sibling package
(Section~\ref{sec:integration}) reflects three distinct concerns:
different release cadence (linguistic models vs.\ OCR pipelines),
different consumers (downstream agents, MCP clients, treebank exporters
--- none of which need OCR), and different dependencies (Stanza's
$\sim$1\,GB of model weights and PyTorch should not be imposed on the
OCR path).

\section{Architecture}
\label{sec:architecture}

\subsection{The pipeline}

Per PDAI document $D$, processing proceeds as in
Figure~\ref{fig:pipeline}.

\begin{figure}[t]
\begin{verbatim}
  Unit Selector  ->  Language Detector  ->  Pipeline Registry
        |                  |                       |
  (sections /          (langid +                 (cached
   letters /            lingua,                  stanza.Pipeline
   catalog /            manifest                 keyed on
   pages)               hints)                   (lang, package))
                                                  |
                                                  v
                  Pre-processors  ->  Stanza run  ->  Post-processors
                   (placeholders)    (tokenize,      (restore phs,
                                      mwt,pos,        page_at_token,
                                      lemma,          MathRef misc)
                                      depparse,
                                      ner)
                                                  |
                                                  v
                       Persist  -> on-disk stanza/units/<id>.{conllu,json}
                                +  per-doc index.duckdb stanza_* tables
                                                  |
                                                  v
                       Aggregator -> global ai-stanza.duckdb
\end{verbatim}
\caption{The \texttt{ai-stanza} pipeline. Language detection precedes
pipeline selection unconditionally.}
\label{fig:pipeline}
\end{figure}

\subsection{Unit kinds}
\label{sec:units}

We recognise four logical unit kinds, dispatched by per-repository
configuration:

\begin{description}
  \item[\texttt{section}] a \texttt{<section>} in the PDAI
    \texttt{structure.xml}, e.g. a \emph{caput} or \emph{propositio} of
    Euler's E57. Preferred when ai-scholar's structure layer is present.
  \item[\texttt{letter}] one piece of correspondence, with
    sender / recipient / date carried as unit metadata. Used for
    Pirckheimer, Euler--Goldbach, von Kármán.
  \item[\texttt{catalog}] one entry in a catalog corpus (sundials,
    ancient metrology).
  \item[\texttt{page}] the OCR page itself --- the universal fallback,
    so \emph{every} pdai document is at minimum parseable.
\end{description}

The \texttt{Unit} dataclass carries the text, a language hint, optional
page span, and a free-form metadata dictionary; the kind is just a
string column in \texttt{stanza\_units}.

\subsection{The schema}
\label{sec:schema}

The per-document and global databases share the schema below
(\texttt{src/ai\_stanza/store/schema.sql} in the package source).

\begin{lstlisting}[language=sqltight]
CREATE TABLE stanza_units (
  unit_id        TEXT PRIMARY KEY,
  doc_id         TEXT NOT NULL,
  unit_kind      TEXT NOT NULL,
  lang           TEXT NOT NULL,
  package        TEXT NOT NULL,
  n_sentences    INTEGER, n_tokens INTEGER,
  page_start     INTEGER, page_end INTEGER,
  text_sha256    TEXT,
  parsed_at      TIMESTAMP,
  parser_version TEXT
);

CREATE TABLE stanza_sentences (
  sent_id     TEXT PRIMARY KEY,
  unit_id     TEXT NOT NULL,
  sent_no     INTEGER, text TEXT,
  page_start  INTEGER, page_end INTEGER
);

CREATE TABLE stanza_tokens (
  token_id  TEXT PRIMARY KEY,
  sent_id   TEXT NOT NULL,
  idx INTEGER, form TEXT, lemma TEXT,
  upos TEXT, xpos TEXT, feats TEXT,
  head INTEGER, deprel TEXT, misc TEXT
);

CREATE TABLE stanza_entities (
  ent_id TEXT PRIMARY KEY, sent_id TEXT,
  start_tok INTEGER, end_tok INTEGER,
  ent_type TEXT, text TEXT
);

-- Denormalised fast lane.
CREATE TABLE stanza_deps (
  sent_id    TEXT NOT NULL,
  head_idx   INTEGER, dep_idx INTEGER, deprel TEXT,
  head_form  TEXT, head_lemma TEXT, head_upos TEXT,
  dep_form   TEXT, dep_lemma  TEXT, dep_upos  TEXT
);

CREATE INDEX idx_tokens_lemma       ON stanza_tokens(lemma);
CREATE INDEX idx_tokens_upos_lemma  ON stanza_tokens(upos, lemma);
CREATE INDEX idx_deps_deprel_lemmas ON stanza_deps(deprel, head_lemma, dep_lemma);
CREATE INDEX idx_entities_type_text ON stanza_entities(ent_type, text);
\end{lstlisting}

The \texttt{stanza\_deps} table is the denormalised fast lane discussed
in Section~\ref{sec:design}. It is populated at parse time from each
non-root token and its head, with both endpoints' \texttt{form, lemma,
upos} duplicated onto the row. Storage cost on a 10\,M-token corpus is
$\approx 300$\,MB additional to the canonical \texttt{stanza\_tokens}
columns; query cost on the dominant SVO shape drops from hundreds of
milliseconds to a few.

\subsection{On-disk \emph{stanza document}}
\label{sec:disk}

Co-located with each PDAI document we write:

\begin{verbatim}
<pdai>/<repo>/<doc>/
  stanza/
    manifest.json
    units/
      <unit_id>.conllu
      <unit_id>.stanza.json
    log/
      <run_id>.jsonl
\end{verbatim}

The CoNLL-U files \cite{conllu-spec} are the standard UD interchange
format; the JSON files are exactly
\texttt{stanza.Document.to\_dict()}, sufficient for round-trip without
re-parsing. Treating the database as a derived materialisation of these
files (rather than the source of truth) keeps the linguistic data
\emph{portable}: any third party can clone the corpus and consume the
\texttt{stanza/} directories with no DuckDB.

\section{Querying}
\label{sec:querying}

\subsection{Lemma + POS}

\begin{lstlisting}[language=sqltight]
SELECT t.form, t.feats, COUNT(*) AS occurrences
FROM stanza_tokens t
WHERE t.lemma = 'aqua' AND t.upos = 'NOUN'
GROUP BY 1, 2
ORDER BY occurrences DESC;
\end{lstlisting}

This catches \emph{aqua, aquae, aquam, aquis, $\ldots$} in one shot, with
case/number distinguishable via \texttt{feats}.

\subsection{Subject--Verb--Object}
\label{sec:svo}

The flagship query: \emph{the moon as the subject of a verb of attraction}
--- the predicate-argument structure that Euler's tidal argument in E57
turns on.

\begin{lstlisting}[language=sqltight]
SELECT s.text, u.doc_id, s.page_start
FROM stanza_deps      d
JOIN stanza_sentences s ON s.sent_id = d.sent_id
JOIN stanza_units     u ON u.unit_id = s.unit_id
WHERE d.deprel    = 'nsubj'
  AND d.dep_lemma = 'luna'
  AND d.head_lemma IN ('attrahere','trahere','agere');
\end{lstlisting}

One indexed scan on \texttt{stanza\_deps}; two indexed lookups for
provenance.

\subsection{Named-entity co-occurrence}

\begin{lstlisting}[language=sqltight]
SELECT a.text AS person, b.text AS place, COUNT(*) c
FROM stanza_entities a
JOIN stanza_entities b ON a.sent_id = b.sent_id
WHERE a.ent_type = 'PER' AND b.ent_type IN ('LOC','GPE')
GROUP BY 1, 2
ORDER BY c DESC LIMIT 50;
\end{lstlisting}

\subsection{Provenance}

Every result row carries \texttt{(repo\_id, doc\_id, unit\_id, sent\_id,
page\_start)}. Downstream agents use this tuple to deep-link back into
\texttt{ai-viewer} for the original page image; the on-disk
\texttt{stanza/units/<unit\_id>.stanza.json} is a single-file return for
``give me the full parse''.

\section{Specimen Corpus}
\label{sec:examples}

We ship a four-specimen example suite under
\texttt{agents/data-ml/ai-stanza/examples/}, each chosen to exercise a
different combination of unit kind, language, and genre:

\begin{description}
  \item[\textsf{euler-E57-de-fluxu/}] five sections from Euler's
    \emph{De fluxu et refluxu maris} (Eneström~E57, 1740). 18th-century
    scientific Latin, the \texttt{proiel} package, with inline
    \texttt{\_\_PB\_\ldots\_\_} page-break milestones and
    \texttt{\_\_MATH\ldots\_\_} formula placeholders.
  \item[\textsf{pirckheimer-letter/}] a Renaissance Latin / German
    mixed-language letter by Willibald Pirckheimer to Albrecht Dürer.
    Exercises mixed-language detection, the letter-as-unit selector, and
    NER on early-modern personal names.
  \item[\textsf{euler-to-goldbach/}] a paragraph from the Euler--Goldbach
    correspondence, exercising the mathematical-historical lexicon
    (\emph{theorema, propositio, primus, summa}) and lemma normalisation
    across letter exchanges.
  \item[\textsf{karman-letter/}] a 1934 letter from Theodore von Kármán
    to Ludwig Prandtl in technical German with English code-switching ---
    20th-century engineering correspondence, NER on organisations
    (NACA, GALCIT, Caltech, Aachen, Göttingen).
\end{description}

The four specimens together demonstrate the same engine carrying classical,
early-modern, and modern register; scientific, epistolary, and technical
genres; one and four named-entity-supporting languages; and the three
unit kinds that account for $> 95\%$ of our corpus.

\section{Integration with Existing Infrastructure}
\label{sec:integration}

\texttt{ai-stanza} sits between two siblings in the umbrella:

\begin{itemize}
  \item \textbf{Reads} from \texttt{ai-scholar} \cite{ai-scholar-arxiv}:
    \texttt{structure.xml} and the per-document \texttt{index.duckdb}
    (\texttt{sections}, \texttt{pages}, \texttt{formulas}). These are
    treated as read-only inputs per the \texttt{structure.xml}
    contract.\footnote{See \texttt{agents/document-processing/ai-scholar/docs/structure-xml-contract.md}.}
  \item \textbf{Writes} additively into the same per-document
    \texttt{index.duckdb}, but only into \texttt{stanza\_*} tables --- no
    overlap with ai-scholar's existing \texttt{sentences} /
    \texttt{tokens} tables. A future migration can unify these once both
    teams agree.
  \item \textbf{Aggregates} into a global
    \texttt{ai-stanza.duckdb}, which feeds the cross-document concept
    layer \texttt{ai-index} \cite{ai-index-arxiv}.
  \item \textbf{Exposes} a read-only MCP server (Model Context Protocol)
    with five tools (\texttt{stanza\_lemma\_search},
    \texttt{stanza\_svo\_search}, \texttt{stanza\_ne\_search},
    \texttt{stanza\_unit\_get}, \texttt{stanza\_doc\_summary}), proxied
    through the umbrella's \texttt{ai-platform-mcp}.
\end{itemize}

The legacy \texttt{ai\_scholar.structure.stanza\_parse} module remains in
place; a follow-up patch will replace its body with a thin delegation to
\texttt{ai\_stanza.parse.runner.parse\_document}, preserving its current
return shape and the Latin/\texttt{proiel} default for backward
compatibility.

\section{Performance}
\label{sec:performance}

Target performance numbers (the runner is in scaffolding as of this
draft; measurements will appear in v0.2):

\begin{table}[h]
\centering
\begin{tabular}{lll}
\toprule
\textbf{Operation} & \textbf{Scale} & \textbf{Target} \\
\midrule
Parse one section (Latin proiel)         & 250 tokens          & $<200$ ms \\
Parse a 360-page document                & one doc, single thread & $<5$ min \\
\texttt{query svo} on global DB          & $10^7$ tokens       & $<5$ ms \\
\texttt{query lemma} with \texttt{upos}  & $10^7$ tokens       & $<5$ ms \\
\texttt{query ne} co-occurrence          & $10^5$ entities     & $<50$ ms \\
\texttt{index sync} of one updated doc   & $10^4$ tokens       & $<1$ s \\
\bottomrule
\end{tabular}
\caption{Target performance numbers on Apple Silicon (M1, 16~GB).
Pipeline construction (model load) is amortised across a batch via the
cached pipeline registry.}
\label{tab:perf}
\end{table}

For a 1\,000-document, 100\,000-page corpus, on-disk artefacts total
$\approx 5$~GB of CoNLL-U+JSON plus $\approx 50$~GB of per-document
DuckDBs and a $\approx 30$~GB global aggregate. This sits comfortably on
a laptop SSD --- a deliberate design constraint, since reproducibility
across sites is one of our acceptance criteria.

\section{Discussion}
\label{sec:discussion}

\subsection{What we gain}

Three things: (i)~\emph{lemma normalisation}, which collapses the
inflectional zoo into a single queryable identity per word; (ii)~\emph{
dependency structure}, which lets us ask grammatical questions
(\emph{who is the agent of this verb?}) directly, not as a brittle
keyword conjunction; (iii)~\emph{millisecond latency on millions of
tokens}, which means the historian can iterate, not batch.

\subsection{What we knowingly trade}

We trade \emph{recall on noisy text}. Stanza was trained on UD treebanks
that are cleaner than 18th-century OCR; lemma and dep accuracy degrade on
heavily OCR-damaged passages. Our mitigation is to gate parsing on
ai-scholar's OCR confidence (planned, v0.2) and to mark low-confidence
units in \texttt{stanza\_units}.

We also trade \emph{LLM-style elasticity}. A vector-embedding query for
``moon-tide causation'' would surface relevant passages even with no
exact lemma match; our SVO query returns nothing if the user does not
know the right verb lemma. We address this by composing: ai-stanza
provides the lemma vocabulary that an LLM agent can query against, with
the LLM proposing the lemma and ai-stanza enforcing the structure.

\subsection{What we leave open}

Mixed-language units beyond dominant-language tagging; cross-document
discourse links (anaphora across letters); and the unification of
ai-scholar's legacy \texttt{sentences}/\texttt{tokens} tables with the
\texttt{stanza\_*} schema. None block the v0.1 design and all are tracked
as open questions in the package's \texttt{PLAN.md}.

\section{Conclusion}
\label{sec:conclusion}

Lemma + dependency parses, stored in a denormalised columnar embedded
database keyed on the four query shapes that scholarly use actually
produces, gives historians a kind of search that neither full-text nor
vector retrieval can: \emph{deterministic, structurally aware, fast,
reproducible}, and trivially shippable on a laptop. \texttt{ai-stanza}
packages this as an autonomous add-on to the PDAI infrastructure, with
a four-genre specimen suite to demonstrate its reach. The combination
of language-first detection, denormalised dep edges, and the on-disk
CoNLL-U+JSON convention is, we argue, the right shape for
linguistic-tree querying over large historical corpora --- and is
deliberately small enough to stay that way.

\bibliographystyle{plain}
\bibliography{references}

\end{document}