FEN × Meta is a human-governed validation layer for knowledge extracted
from documents by an AI model. The model (Meta muse-spark-1.3)
reads two or three source documents and proposes structured claims.
Deterministic code — no model involved — checks that every quoted piece of
evidence really exists in its source. A human reviewer then accepts,
rejects, or disputes each proposal. Only that human decision creates a
final status, and every decision is recorded with its full provenance:
source document, exact quote, model and prompt, reviewer, verdict, and time.
Open the simplified demo, leave the pre-filled synthetic documents (or switch to the Sumerian dictionary entries), and click Analyse documents. You will get proposals labelled “Proposal — awaiting human decision”. Click Show in source to see a quote highlighted in its document, then record an Accept on one proposal and a Dispute on the contradiction. The accepted item appears in the Registry with a downloadable record. The whole path takes about a minute.
Three ways, all equivalent: paste text into the document fields; click
+ Add third document for a third source; or use
Upload document(s)… / drag-and-drop with .txt,
.md, or .json files. A JSON file shaped as
{"documents":[{"title": "...", "text": "..."}]} expands into
several documents. Files are read in your browser and placed into the
fields — nothing is sent anywhere until you click Analyse. Limits: at most
3 documents and 30,000 characters in total (the same limit the analysis
API enforces). Over-limit text is rejected with an explanation, never
silently truncated, because truncation would break quote verification.
Demo (synthetic) loads an invented neutral example, labelled
synthetic so it can never be mistaken for corpus data.
Sumerian dictionary entries (ePSD2) loads short, verified dictionary
excerpts for the candidate lemmas lu, bahar, and
me. They are dictionary excerpts — not corpus records and not
gold evaluation data. Gold data is created only from attested source texts
selected and verified by a human.
d1, d2, d3 in the demo).| Button | What you are saying | What happens next |
|---|---|---|
| Accept | “This claim faithfully represents its source, and I take responsibility for it.” | Status becomes Accepted. The item enters the Registry as accepted knowledge and can be downloaded as a JSON-LD record. Available only when the quote is verified — otherwise the button is disabled and the API returns 409. |
| Reject | “This must not become knowledge” — wrong, invented, or not supported by this source. | Status becomes Rejected. The record is not deleted: it stays in the decision history and provenance with your comment, so it remains visible that the community already considered and rejected it. It never enters the Registry. |
| Dispute | “I disagree or I am not sure — this needs discussion or an expert.” | Status becomes Disputed. There is no final outcome: the item stays open for another reviewer or an expert. This is the normal verdict for contradictions between sources. |
Every line is a decision that was already recorded (not a button), in one format:
[Final status] · [reviewer] · [proposal id] — [your comment] · [time, UTC]
Example: “Rejected by a human reviewer · reviewer-1 · c3 — Plurality
was expressed by reduplication…” means proposal c3 was
rejected with that comment. A contradiction such as x1 marked
Disputed was deliberately left open instead of being decided on the spot.
Yes — decide again on the same item. Decisions are immutable and append-only: the new verdict becomes a new record, the old one stays in the history untouched, and the item's current status is always the latest decision. Nothing is ever edited or silently overwritten.
Deliberately. A proposal is the model's output; editing it would destroy the audit trail (nobody could later tell what the model said versus what a human wrote). If a proposal is almost right, reject or dispute it with a comment explaining the correction, or submit a corrected document and analyse again — both actions stay visible in the record.
| Status | Meaning | Who can set it |
|---|---|---|
| Proposed | The model suggested it; nobody has judged it yet. This is the only status a fresh analysis can have. | The model (by proposing) |
| Accepted | A human accepted it; it is in the Registry. | Human only |
| Rejected | A human rejected it; kept in the record, not in the Registry. | Human only |
| Disputed | A human flagged disagreement; awaiting discussion or an expert. | Human only |
The architectural rule of the whole system: the model only proposes.
No code path can assign accepted without a recorded human
decision behind it. This follows the project's existing decision ADR-004.
That the quote can be located in the referenced source document: an exact substring match, or a match after whitespace is normalised (so line breaks and multiple spaces do not matter). The check is deterministic code; the model is not asked and cannot influence it. Quote verification proves traceability — the words are really in the source. It does not prove the claim is true, and it does not prove the model understood the quote correctly. Judging meaning remains your job as the reviewer.
The proposal's quote could not be located in its source — the classic
case of a model inventing evidence. The card is shown red, states
“Quote NOT verified”, and Accept is disabled. The API enforces the
same rule server-side (409), so bypassing the button in the
interface does not help. You can still Reject or Dispute the item — that
is usually the right response to an invented quote.
It opens the proposal's source document with the exact quote highlighted. Use it before deciding: it is the fastest way to see the evidence in context instead of trusting the card. If the quote cannot be located, the panel tells you so — that is precisely why acceptance is disabled for that item.
Usual causes: the quote was edited in the document field after analysis (re-run Analyse); punctuation or case differs (matching normalises whitespace, not punctuation); or the quote spans text that the model joined from two places. Check with Show in source, then either fix the document and re-analyse, or Reject with a comment.
A contradiction card appears when the model thinks two claims may
conflict. It shows both claims side by side with their quotes, a
kind (for example temporal_mismatch,
value_mismatch, direct_negation, or
other), the model's explanation, and
alternative interpretations — readings under which the conflict would
dissolve (“the dates may refer to different events”, and similar).
Dispute is the default: you are recording that the sources disagree and a person with domain knowledge must resolve it. Accept on a contradiction means “yes, this is a genuine conflict worth keeping as a documented disagreement” (still only possible when the involved quotes are verified). Reject means the model invented the conflict — for example it misread one of the quotes, which you can check with Show in source on both claims.
On proposals that name one of the verified Sumerian candidate lemmas, the card shows a green dictionary badge, for example: Dictionary (ePSD2): lu [PERSON] (N) · person; man · 18,895 attestations (CC BY-SA 3.0), linking to the dictionary entry.
It proves the lemma exists in the electronic Pennsylvania Sumerian Dictionary with that gloss and that many corpus occurrences. It is orientation metadata for the reviewer. It does not verify the proposal's quote (that is the separate quote check), it is not evidence for the claim, and it never counts as a vote.
Because ePSD2 has two different headwords written me:
me [BEING] (noun — Being, divine properties) and
me [BE] (verb — to be). The badge follows the gloss in the
proposal so the two are never merged. The same distinction applies in the
lemma candidate register: noun and verb are separate entries.
By design. Case endings, pronouns and verbal affixes are grammatical markers, not lemmas. Recording them as dataset lemmas would add noise to lemma-extraction evaluation. They belong in grammar references, not in the candidate register.
The demo switcher offers three presets. Switching replaces the documents in the form; everything else — analyse, quote check, decisions, registry — is the same code path. One engine, three data sets.
| Preset | What it contains | What it is for |
|---|---|---|
| Demo (synthetic) | An invented trade-route story | Nothing to licence; the default |
| Sumerian dictionary entries (ePSD2) | Verified dictionary excerpts for
lu, bahar and me (CC BY-SA 3.0) |
Lemma extraction and the dictionary badges |
| EU AI Act — public texts (not legal advice) | Articles 11 and 18 of Regulation (EU) 2024/1689 from EUR-Lex, plus one SYNTHETIC commentary paragraph | A wrong-attribution case in a regulated domain |
Because verification answers one question only: do these exact words appear in the document this claim points at? It does not answer “is this true”, “is this authoritative”, or “is the attribution correct” — those are human judgements. The provenance badge is the second, independent signal: it says the document itself is synthetic, secondary, or primary. A verified quote from a synthetic source is a located sentence and nothing more.
The quote really is in the document, but the model itself flagged its reading as
high or insufficient_evidence. The card then shows this
warning next to the green “Quote verified” badge, because location and meaning are
different things: the words exist, the conclusion drawn from them is not verified.
Read the quote and decide yourself — Accept stays enabled on purpose, since the
system does not pretend to judge meaning.
In Article 18 (“Documentation keeping”). Article 11 (“Technical
documentation”) contains no ten-year period at all: it requires the documentation to
be drawn up before the system is placed on the market and kept up to date. The
dataset keeps the official wording, and a test asserts that 10 years
appears in the Article 18 text and not in the Article 11 text — so the demo
case cannot be quietly faked by editing the documents.
No. In one live run the model surfaced that contradiction on its own; it is a
model-found item, and gold_contradictions in the dataset is
intentionally empty because no human has verified it as a gold label. The model may
or may not surface it on another run — do not treat it as guaranteed.
No, on both counts. The texts are public regulation reproduced with attribution (EUR-Lex reuse policy, Commission Decision 2011/833/EU) and labelled “Not legal advice” in the UI. Nothing here claims that an AI system complies with the AI Act, and the demo does not assess conformity with any article.
Yes: upload a file or type into the fields. Documents you supply are treated as
primary by default, so no provenance badge appears — which is honest
only if they really are primary sources. The API accepts provenance
per document (primary, secondary,
synthetic); if you paste a commentary or a summary, say so.
The complete chain for one item: the source document, the exact quote, the model proposal (model name, SHA-256 prompt hash, timestamp), and every human decision in order (reviewer, verdict, comment, time). If you can see this chain, you can audit how the claim got its status without trusting anyone's summary of it.
Into the Registry, and into durable storage. Accepted items are listed
with their statement, quote, reviewer, time and the provenance of the source
document, and each can be downloaded as a JSON-LD record
(GET /records/{item_id}) containing the claim, quote, source
documents, model and prompt hash, and all decisions — the artefact a knowledge
graph can later ingest unchanged.
Storage has two layers. Locally, analyses and decisions are appended to a log
(artifacts/store.jsonl), so restarting the demo does not lose your
verdicts. With SUPABASE_DB_URL set, the same analyses, documents,
proposals and decisions are written to the Supabase database (schema in
supabase/migrations/), and the public Registry page reads the
accepted-knowledge view from there. Writing into the baseline Kafka → RDF
governance pipeline exists in the underlying research prototype but is
deliberately not claimed as wired into this hackathon workflow.
Public: the view registry_public — accepted items only, with the
statement, quote, reviewer, verdict, time, model, prompt hash, quote-verified flag
and source provenance. Private: source-document texts, proposals that no human has
decided, and the decisions table itself. The frontend uses a
publishable key that is safe to ship precisely because row-level security grants it
SELECT on that one view and nothing else — reading a base table with
it returns 401. The page re-reads the view every 20 seconds, so a
decision recorded in the demo appears there without a reload.
No. Decisions are append-only, and the database enforces it: a trigger refuses
UPDATE and DELETE on the decisions table. Deciding again
inserts a new row, and the item's current status is resolved from the latest
decision. The recorded history therefore shows every verdict, including a later
correction — it is not a rewrite of the past.
A gfen:ValidationRecord: item and analysis identifiers,
current status, the claim object (quote, statement, interpretation,
uncertainty), source documents, the model proposal metadata (model,
prompt hash, creation time), and the full decision list. It is designed
to be self-contained: a third party can verify the chain from the file
alone.
Because a record is a decision artefact. The API returns
409 for an undecided item: a model proposal on its own must
never be exportable as if it were validated knowledge.
The classic portal and Triadic view are the pre-hackathon baseline demo: candidates are voted on by a simulated community DAO. Their outcome buttons mean the same as the Review verdicts:
| Candidates button | Equivalent verdict | Meaning |
|---|---|---|
validated | Accept | Include this entity as community-validated. |
disputed | Dispute | Flag disagreement; needs discussion or experts. |
rejected | Reject | Do not include; the rejection stays on record. |
Click “Details — what am I voting on?” in the candidates list (or in the Triadic consensus list). The detail page shows the entity, document, current votes and quorum progress, peer assessments with comments, the recorded decision if one exists, the governance registry record, an explanation of each outcome, and the same voting buttons — so you can decide on the page where you read the evidence.
These belong to the simulated DAO mechanics of the baseline demo. Quorum: in community mode a candidate is decided when enough votes arrive and the majority wins. Quadratic Voting (QV): a vote has an intensity of 1–5 and costs intensity² credits, so strong opinions are possible but expensive; a decision is taken when one outcome's weighted score reaches the threshold. Delegation (liquid democracy): a voter who has not voted can delegate their weight to another voter, and it follows that voter's choice. These mechanisms are a simulation for the demo (ADR-005); production governance lives outside this repository.
They download the candidate's governance record from the registry in different formats: Turtle, JSON-LD, N-Triples, or an RO-Crate package. They are read-only exports of what was decided, not editing tools.
| What you see | What it means | What to do |
|---|---|---|
| “The model did not respond…” (502) | The model provider is unreachable, not configured, or returned output that failed the contract after one format retry. Nothing was saved and no proposal was invented as a fallback. | On the hosted demo, retry once — a slow model call can also hit the 60-second function limit (504). Running locally, check that scripts/start_demo.py is up and the provider keys are set. |
| 422 on Analyse | Invalid input: no documents, duplicate document ids, or more than 30,000 characters in total. The request is rejected before any model call. | Fix the documents; split long texts instead of expecting truncation. |
| 409 when accepting | You tried to accept an item whose quote is not verified — or to export a record for an undecided item. | Check the quote with Show in source; Reject or Dispute instead, or fix the document and re-analyse. |
| 404 on provenance / record | The item id is unknown to the current registry — often an item from an earlier session after a restart without persistence. | Run the analysis again in this session, or start via start_demo.py so the store persists. |
| “Registry unavailable” | The demo could not reach the local API for the Registry section. Proposals and voting are unaffected once the backend is up. | Start (or restart) the local demo stack. |
| Analyse works locally but not on the public website | The hosted page is the static interface; the model backend runs locally and is not public. | Run the local demo for analysis; the public page is for reading and presentation. |
Uploads are read in your browser into the document fields. Text is sent to the model service only when you click Analyse, as the analysis request. API keys are never entered in the interface and never appear in responses, records, or logs; they come from environment variables on the backend machine only.
Labels are explicit everywhere: synthetic documents and fixtures are invented neutral examples; the Sumerian material is short dictionary excerpts from ePSD2 for verified candidate lemmas; real corpus texts and gold evaluation data are supplied and licence-checked by a human and are never generated by the agent. Model responses used in tests are marked SYNTHETIC or RECORDED in their fixture files.
ePSD2/Oracc data is licensed CC BY-SA 3.0: reuse is allowed with
attribution, and substantial reuses must keep the same licence. The
canonical attribution is recorded in eval/dataset/SOURCES.md
together with the other sources' terms (notably: ETCSL texts are not
openly licensed and are not republished here).
| Term | Meaning in this system |
|---|---|
| Claim | One atomic statement extracted from a source, carried by an exact quote. |
| Quote | The verbatim evidence span copied from the source document; the unit of verification. |
| Interpretation | The model's reading of a claim, stored in a separate field so it can never masquerade as the source's words. |
| Proposal | Any model output before a human decision. Proposals have no authority. |
| Provenance | The auditable chain: document → quote → model proposal → human decision(s). |
| Prompt hash | SHA-256 of the exact prompts that produced a proposal, so the run can be reproduced and audited. |
| Lemma | A dictionary headword (for example lu [person]), independent of its spellings in texts. |
| Attestation | A real occurrence of a form in a corpus text, with a citable source. |
| Gold data | Human-made reference answers for evaluation. Agents never generate gold data here. |
| B0 baseline | The share of proposals with unverified quotes — what would have passed as trustworthy without the verification gate. |
| Registry | The store of recorded decisions and accepted knowledge (a local log in the demo launcher; the Supabase database when configured). |
| Source provenance | Whether the document a claim cites is a primary text, secondary commentary, or a synthetic illustration. Shown as a badge; never inferred from the claim itself. |
| SYNTHETIC source | A document written for the demo, not a real publication. Its quotes can be verified (they exist in that document) and still prove nothing about the world. |
It reproduces Articles 11 and 18 of Regulation (EU) 2024/1689 with attribution and marks a synthetic commentary as synthetic. It does not give legal advice, does not interpret the Act, and does not claim that any AI system complies with it. The third document is illustrative and is labelled as such in the UI; its quotes are verifiable because the words are in that document, which is exactly why the provenance badge exists.
Contradiction detection is a model capability, not a deterministic check. In a
live run the model surfaced the Article 18 vs commentary contradiction; on another
run it may not. The dataset's gold_contradictions list is empty
because no human verified that item as a gold label. Treat model-found items as
proposals like any other.