Case study
ProofMart
In developmentA verdict nobody can audit is a rumour with a confidence score.
The problem, in the user's words
"The tool told me the invoice was 87% likely to be tampered with. My client asked me where. I could not answer, so I sent the file back and lost the account."
The people checking documents — small lenders, campus admissions desks, freelance verifiers — are not fraud analysts. They cannot defend a number they cannot point at. What they need is not a better score; it is a place on the page to put a finger.
Why the obvious solution fails
The obvious build is a classifier: features in, probability out, threshold at 0.8. It is fast to build and impossible to argue with, which sounds like the same thing but is the opposite. A single probability collapses ten independent signals — font substitution, recompression artifacts, inconsistent object generations, mismatched metadata — into one number that loses the only information the user actually needed: which signal, and where.
It also cannot be re-checked. If the model version changes, last month's verdict silently means something different, and there is no artifact proving what was true at the time of the decision.
Architecture — one diagram
The pipeline is upload → classify → extract → OCR → verify. Verification runs six deterministic markers against the extracted content; each one is a typed finding with a page index, a bounding box, a severity and the raw evidence that produced it. The verdict is derived last, from findings, using fixed, explainable precedence — never a blended probability.
Findings are append-only. The dossier is the Ed25519-signed serialisation of the evidence index plus the marker versions that produced it.
Three decisions worth defending
Coordinates are part of the schema, not a nice-to-have
A finding without a page index and bounding box fails validation and never reaches the index. That single constraint is what forces every marker to be explainable — a marker that cannot say where is a marker that cannot ship.
The dossier is signed and detached from the API response
Verdicts travel as an Ed25519-signed artifact, so a third party can verify the result without trusting my server or my uptime. It also makes the record immutable: re-running a newer marker produces a new dossier rather than quietly rewriting the old one.
Marker versions are pinned in the output
Every dossier carries the exact version of every marker that ran. Model and heuristic drift is real; without pinning, an old verdict becomes unreproducible the first time a threshold is tuned.
What shipped
~9.2k lines of tested TypeScript across 15 Vitest suites and 4 Playwright end-to-end specs. The upload-through-verify pipeline, all six markers, the evidence index, Ed25519 dossier signing, Supabase Auth with row-level security and rate-limited API keys are working end to end. The public API docs page and the hosted Evidence Rail viewer are not done — which is why the badge reads IN DEVELOPMENT and not SHIPPED.
What I'd do differently
I built the markers before the evidence schema and paid for it twice: the first two had to be rewritten once findings became typed records with coordinates. The schema was the product; I treated it as plumbing.
I would also have kept the surface smaller. Six markers with a working viewer would have been a more honest 70% than six markers, a partial API and no viewer.