Building a Sourced Reference Corpus With Agentic Data Mining

An AI agent did the volume; a copyright firewall, schema gates, and an adversarial red team did the rest - and the piece ships its own charts from the real data to prove it.

The Laundering Problem

Point a capable AI agent at a shelf of reference books and ask it to build you a dataset, and you will get one. You will also, usually, get a quiet copyright problem: most AI research pipelines read copyrighted prose and let its phrasing seep into their output, so what comes out the other side is a derivative work wearing a structured-data costume. It looks like facts. It carries someone else's sentences.

We wanted the opposite, and we wanted it to be provable rather than promised: a specialist reference corpus that carries facts and our own authored meaning only, and never a source's words, built mostly by an agent working at a volume no human would sustain. The subject is a performing-arts vocabulary with rival codified schools - a good stress test, because the terms are contested, the best references are under copyright, and the schools genuinely disagree. The domain is incidental to the point. What follows is the machinery, because the machinery is the argument: the discipline is built into the pipeline, not trusted to the model.

What Got Built

Before the discipline, the shape of the thing. The corpus is 1,682 terms drawn from 39 sources, carrying 4,059 individual attestations - a term is not a headword and a blurb but a record of many attested facts, each traceable to who said it and where. Most terms are compounds and atoms built up from a small base vocabulary rather than a flat list of names, which is what you would expect of a domain that composes meaning rather than enumerating it.

The term space, decomposed. The corpus split by structural kind - atoms, compositions, lexicalised phrases, and the still-unclassified - each block coloured by how well corroborated it is. Read it as: the domain is mostly built, not listed, and the fully corroborated share is still a minority. This is the shape of a corpus mid-build, and it says so.

That refusal to overstate is the first design choice worth noticing. A term carries whatever has been attested about it - a written gloss, a syllabus grade, a function type, an etymology, a pronunciation - and the record shows which of those facets it actually has, rather than implying completeness it has not earned.

What a term carries. For each kind of enrichment, the share of the 1,682 terms that have it. Read it as: an entry is a record with many attested facets, and the corpus does not pretend every term is finished.

The Firewall

The rule is simple to state and easy to violate by accident: store what a source contributes, never how it said it. Every attestation is an authored paraphrase in our own words, tagged with its kind - definition, usage, or listing - its source, and a locator precise enough to check. The source's own sentences are never stored in the term record.

Stating a rule is not enforcing one, so the rule is enforced by code at the publish boundary. When a release cut is assembled, a rights step drops any un-re-authored source-derived text - today, the etymology entries - whose source is not cleared to quote, so a published cut can withhold prose that a read of the working tree would still see. Then a publish gate makes the assertion: it breaks every published prose field into overlapping eight-word runs and fails the entire cut if any run also appears in the committed raw text of a source that is not cleared to quote, naming the term, the field, and the source it collided with.

Be precise about what that gate does and does not do, because that precision is the selling point. It cannot prove a paraphrase exists - nothing mechanical can. What it can prove is the negative: that no published sentence overlaps the committed text of a restricted source. It has edges, and they are worth stating - a source whose raw is not held as machine-readable text is flagged uncheckable rather than waved through, and the independent benchmark the corpus measures itself against is deliberately walled off from the gate and covered by a separate advisory audit. Within those edges the guarantee is real: as the gate's own comment puts it, a paraphrase cannot be proven mechanically, but copied prose can be caught mechanically, and catching the copying is enough to keep the output clean by construction. It is an engineering control, not a legal opinion.

There is one principled exception, and it is handled by construction rather than by trust. Public-domain and open-licence sources may be quoted, so a separate concordance keeps a couple of verbatim snippets per term from those sources only - in its own file, never in the term record - and the publish gate exempts exactly those sources from its overlap check. Quotable text lives in one clearly walled place; everything else is paraphrase or it does not ship.

Provenance as flow. Every mined attestation traced left to right, from the extraction pipeline that produced it, through its attestation kind, into the corroboration tier of the term it supports. Read it as: you can follow how a raw capture becomes a corroborated definition, and every strand has a source at its left edge.

Provenance is only worth claiming if it is checkable, and checkability is not uniform. An attestation mined from a spoken-video source through automatic transcription lands on a precise timestamp; one lifted from a dictionary's clean text often resolves only to a page. Of the 4,059 attestations, 1,837 carry a resolvable deep link back to the evidence - an exact timestamp for spoken video, a search-inside query for an archived book - and the gap between pipelines is stark.

Citation reach by pipeline. For each extraction pipeline, the share of its attestations that carry a resolvable deep link back to the evidence. Read it as: spoken-video citations are almost entirely deep-linkable; clean-text dictionary citations far less so. How checkable the evidence is depends on where it came from, and the corpus records the difference rather than averaging it away.

One of those pipelines is an upstream supplier, mined facts-only at a pinned commit: the transcript text itself is never stored, only the facts and their locators, and the exact commit the facts were taken at is recorded in the release manifest. The trust chain reads the same way at every link - a pinned, facts-only dependency feeding a gated, facts-only corpus.

Building Meaning Instead of Asserting It

A definition is where a corpus is most tempted to launder authority. The easy move is to copy the clearest source and move on. The disciplined move is to build the meaning: corroborate a term across independent sources, weight the evidence by trust rather than count it, and only then author a single definition in one voice.

The weighting is the mechanism that stops a popular error from becoming a fact. Each definition attestation carries a trust weight - three for an authoritative, independently authored source, two for a solid independent reference, one for a derivative one - and, crucially, sources that share a lineage do not stack: the corpus keeps the strongest weight per lineage and sums those, so five books copying one ancestor count once, at their best. A term clears to corroborated only when that lineage-deduplicated trust reaches a set threshold; with weaker or usage-only evidence it is merely sourced; with no attested meaning at all - neither a definition nor a usage - it is a stub. Status falls out of the weight and independence of the evidence, not the raw number of mentions.

Corroboration, not counting. The distribution of each term's trust-weighted corroboration score, drawn separately for the three status tiers, with the median marked. Read it as: the corroborated tier separates cleanly at its threshold and the stubs sit at zero, because status is earned from evidence weight rather than imposed by a mention count.

Meaning is built in stages, and the corpus is candid about how few terms have travelled the whole way. A term starts by merely existing, gains a source, gains an authored gloss, becomes robustly corroborated, and - for a small minority - earns a long-form treatment. The funnel narrows hard, which is the point: a lot of terms, a few genuinely finished.

The term maturity funnel. How far terms progress as meaning is built - in the corpus, sourced, glossed, corroborated, given a long-form treatment. Read it as: the authoring pipeline made visible, with the attrition shown rather than hidden. (The stages are increasing investment, not strict subsets; the glossed stage here counts a robust authored gloss, a stricter bar than a raw gloss count.)

When the rival schools genuinely disagree, the disagreement is kept as its own attested claim rather than flattened into a false consensus. Each school can be profiled across the same measures and held side by side, so the corpus represents several traditions at once instead of picking a house view.

Coverage across traditions. Three schools profiled across five measures, each axis scaled to its own maximum so the shapes compare. Read it as: several rival traditions held at once and each measured, rather than one flattened into the others. (Each axis has its own scale; the schools are shown as neutral labels.)

The authored definition, once written, is stamped with the exact set of sources it was written against. When a later source changes that set, the term is flagged as stale and queued for review - so the corpus knows when its own prose has fallen behind its own evidence. At this cut, exactly one gloss carries that flag.

The Discipline That Makes the AI Safe

This is the part that makes an agent trustworthy at volume: not asking it to be careful, but making a careless result fail loudly. The agent proposes; the guardrails dispose - and, crucially, dispose differently by risk.

Take the cross-reference hotlinks the agent proposes - terms named inside another term's prose, marked so a reader can jump between them. The discipline here is not blanket human review; it is default-deny on the risky class. A form that reads as high-precision - foreign, or multi-word - auto-links and ships, because it is unlikely to point at the wrong thing. The ambiguous single-word forms, the ones that could hotlink an everyday word to a technical term, are held back by default: of that reviewed class a human has confirmed only 49 and denied 283, with the rest still waiting. Automation where it is provably safe, a default-deny gate where it is not.

The review gate. How the agent's proposed cross-reference hotlinks are disposed of - auto-linked, in review, denied, human-confirmed. Read it as: the provably safe forms auto-link and ship, but the ambiguous single-word class is default-denied to review, where humans confirm few. Default-deny on the risky class, not blanket automation.

Two standing mechanisms keep the data itself sound. The first is structural: the data is the artifact and the human-readable markdown is a generated view, so every write is schema-validated the moment it happens. A validate-on-write hook runs on each edit and rejects a bad one immediately, feeding the error straight back to the editing session as a correction rather than letting it settle into the corpus. The second is a standing defect guard that scans for the fingerprints of bad extraction - glued-together words, transcription mojibake, stray locator noise, leaked running headers - and attributes each finding to the source behind the term's definition, so the fix happens at origin, in the extractor, not as a patch in the corpus. It reports; it does not edit.

And before a release cut, authored prose is put through an adversarial red team - a skeptic pass whose job is to find factual defects, not to rubber-stamp. This is a process, not a line of code, and it is worth saying so plainly: it runs before release cuts, not automatically on every one. But its effect is on the record. One cut was a near-pure remediation release whose changes were fixes, not new content: sixteen confirmed factual defects a red-team pass had caught - a miscounted repetition, a wrong travel direction, a wrong material in a list, a garbled etymology, and more. Another cut added a hundred and forty-seven red-teamed glosses in one pass. A release whose entire purpose is to fix what an adversary found is a strong anti-hype signal - the kind of thing you cannot fake.

Watching It Hold Up

None of this is worth much if you cannot see the state of the data while sourcing continues, so the corpus renders its own health from generated data in a build-free dashboard - coverage against a benchmark, corroboration status, contribution per source, the open work queues. Coverage is the number most likely to be quoted and most easily fudged, so the dashboard refuses to pick a single denominator.

The gauge that earns its number. Coverage shown three ways at once as concentric rings - against the in-scope benchmark, against the whole benchmark, and the share of the corpus with at least one source. Read it as: a coverage figure depends entirely on the denominator you agree to count, so the right move is to show all three and let each be audited.

The corpus is also, underneath, a connected structure rather than a list: terms depend on other terms, and the dependency graph is the closest thing to a map of the domain the data holds. Shown in full, it is dense - the real picture of how interlinked the vocabulary is.

The relation graph, in full. Each node a term coloured by function, each edge a learn-this-before-that dependency. Read it as: the raw prerequisite structure, dense on purpose - a glossary would be a list; this is a network.

The same structure, laid on a ring with the dependencies drawn as chords, is easier to read without losing anything - the hairball becomes a diagram, and the density becomes legible.

The same relations, made legible. The identical terms and dependencies arranged on a ring, edges arcing across as chords. Read it as: the same network as above, ordered so the structure reads rather than tangles. Two views of one truth: the raw shape, and the shape made readable.

What This Buys

Strip the figures away and the through-line is a single claim, now shown rather than asserted: an AI agent can do the volume of a data-mining project and still produce something you own outright, if the discipline lives in the pipeline rather than in the model's good intentions. The mechanisms are the deliverable - a copyright firewall that catches copied prose mechanically, a schema gate that rejects a bad write at the moment it happens, trust-weighted corroboration that refuses to let copies vote twice, an adversarial red team that ships remediation releases, and a dashboard that will not quote a coverage number without its denominator.

None of it claims perfection. The corpus tracks 647 stubs it has not finished, flags its own stale prose, and ships releases whose only content is fixing what an adversary caught. That candour is not a weakness in the story; it is the story. Disciplined AI work does not look like a confident machine. It looks like a machine whose mistakes have somewhere to fail safely, and a record open enough to show them.