Strata — Product Requirements Document

An agent organization that reads the full git history of a repository and turns it into research data, a ten-part essay series, a browser game, visualizations, and a set of live, self-refreshing surfaces. Mine once, read everywhere.

This is the master document for the whole system. Each team also has its own deeper spec (docs/METHODOLOGY.md for the Classification Lab, docs/ATLAS-PRD.md, docs/COMPASS-PRD.md); the operator guide is README.md; the surface-integration brief is INTEGRATION.md. This PRD is the thing that explains what Strata is, why it holds together, and the single discipline that makes it trustworthy.

1. Executive summary

A version-controlled repository is an unusually honest object. Every change it ever underwent is dated, attributed, and recoverable — including the work that failed, the arrangements that were abandoned, and the decisions that were later reversed. Most tools read a repository at HEAD, as a finished artifact. Strata reads it as a record of its own making, and recovers the things the finished surface erases: the hidden labor, the classification scheme argued into shape over time, the abandoned intentions, the reasoning behind the choices that stuck.

Strata is organized as eight agent teams over one shared substrate, the way a real research organization shares a library. Each team reads the same normalized history and diverges only at the transform step — one record, many readings. The output is not a report but a living museum: ten essays and six interactive surfaces, each hydrated from real run data, each refusing to state a number that isn't true.

The whole system rests on one rule, stated once here and enforced everywhere below:

Every figure Strata publishes is real, traced to a commit, or it is not published. Where the instrument is blind, it says so — loudly, on the record.

As of the canonical run (full history, 31 May 2026; 1,360 commits): all eight teams are built, the "planned" row is empty, and the surfaces carry live counts — 71 areas excavated, 956 regrading events, 42 dead threads, 36 grounded decisions, 1 recurring arc.

2. Problem & thesis

The thesis

A finished thing hides the work that made it. Bowker and Star, Sorting Things Out (1999): infrastructure becomes invisible precisely when it works. A shipped feature shows the option that won, never the deliberation; a clean directory tree shows the current filing scheme, never the torque that produced it; a quiet module reads as either finished or abandoned with no way to tell which. The repository contains the erased history — dated and attributable — but nothing reads it back. Strata is the instrument that does.

The problem, concretely

• "Why is the schema like this?" has no answer but archaeology six months on.
• "What did we keep starting and dropping?" lives scattered across dead branches no one revisits.
• "How much work is actually behind this feature?" is invisible once the feature ships.
• "Is this module done, or just dormant?" looks identical from HEAD.

Each of these is recorded in the history and unreadable without an instrument. Strata builds one instrument per question, and makes each answer a public, browsable surface.

3. Audiences & surfaces

Strata serves two audiences from one body of work.

• The reader (public) gets a guided experience: a ten-essay series at /essays/strata and six interactive surfaces (below). No login; everything is statically rendered from committed data.
• The maintainer (you) gets a self-refreshing system: the agents re-run weekly and on demand, land their refreshed data as a reviewable PR, and keep every surface current as the repository grows — with an in-CI Claude available for autofix.

4. The substrate (Bedrock) — read this first

Everything composes because everything reads the same record.

Bedrock (agents/mine.js; the Surveyor and Archivist) mines the full history — commits, file events, and, when a token is present, pull requests — into a normalized record. The schema (schema/schema.sql) is the contract: commits · files · file_events · pull_requests · pr_commits, plus agent-owned derived tables added by each team as it is built (decisions → Atlas, dead_threads → Games, classification_events → Lab, intent_signals → Compass). No team re-mines; they read Bedrock and write their verdicts back beside it, so a later team can read an earlier team's conclusions.

Two properties make the substrate trustworthy:

1. Determinism. Every agent is a pure function of (repository, declared parameters). Re-running on the same commit range reproduces the dataset exactly. Parameters are versioned constants, not buried magic numbers.
2. Source-free outputs. The committed run data carries only structural fields — paths, counts, dates, subjects, SHAs — never file contents. The surfaces and the digest are safe to commit and publish by construction.

5. The eight teams

Each team is one question, a small set of named agents, and a real metric from the canonical run.

5.1 Excavation — the hidden labor

The Excavator surfaces the invisible work behind a shipped feature; the Biographer tells the whole life of one load-bearing file; the Editor shapes the ten-part series. The canonical spotlight: the-beautiful-unfinished/bibliography — 1,377 lines written in a single day, 720 surviving, the rest overwritten with no deletion ever logged because the page is generated. The workhorse: CLAUDE.md — 27 commits in a month, almost none about it. Feeds docs/SERIES.md.

5.2 Classification Lab — the filing scheme

The doctoral core. It treats the directory tree as a classification scheme and records every regrading event (eight kinds, two families) over the whole history, plus longitudinal metrics — breadth, depth, and residual load (the misc/tmp/legacy fraction, Bowker and Star's residual categories made continuous). Method, typology, and honest detector limits are specified in docs/METHODOLOGY.md, which the agent (regrader.js) operationalizes verbatim — if a definition changes there, the constant changes here, and the dataset stays reproducible. Canonical run: 956 regrading events.

5.3 Games — the abandoned (the Graveyard)

The Necromancer finds features begun and dropped and scores their deadness; the Loremaster turns each into a Dungeons & Dragons encounter with a stat block mapped deterministically from git metrics, an epitaph from its real last commit, and a resurrect-or-bury choice. Canonical run: 42 dead threads. The honest limit (shared with the Curator): telling abandoned from finished-and-stable is the same hard problem, so the score leans on signals of incompleteness.

5.4 Curator — the finished (the Collection)

The Graveyard's twin, run on the same doneness detector, returning the opposite verdict: a quiet subtree is either a grave or an exhibit, and the difference is doneness (substantial, settled, clean, still standing). The Curator acquires the finished ones; the Docent writes each a wall label from real git facts. Surface: /strata/museum.

5.5 Cartography — the repository over time (the City)

The Cartographer renders the whole repository as a city that rises commit by commit — districts are top-level directories, buildings are files, floors are size — scrubable across the full history. Auto-districts deeper when one directory dominates. Surface: /strata/city.

5.6 Atlas — the why

The Rationale Miner extracts typed decisions and their stated rationale from commits and PR bodies (a declared five-kind typology: replacement, removal, introduction, reversal, constraint); the Linker binds each to the area it governs via the Excavator's exact area model; the Oracle serves a grounded, searchable surface. Canonical run: 36 decisions logged of 132 detected (confidence ≥ 60), 72 % carrying a verbatim rationale — the headline honesty metric. A decision with no recorded why is logged rationale: null; it is never inferred. Spec: docs/ATLAS-PRD.md. Surface: /strata/atlas.

5.7 Compass — what you keep circling

The only team that points at the future, and therefore the most strictly governed: Compass never predicts. It groups real, dated signals — abandoned branches, re-approach decisions, dead-end clusters — into arcs (recurring intentions) behind a hard evidence floor (≥ 3 signals ∧ strength ≥ 60), frames each one neutrally (a description of evidence, never an imperative), and is allowed — encouraged — to surface nothing when nothing clears the bar. Canonical run: 1 arc surfaced of 16 candidates ("crss"). No model, ever. Spec: docs/COMPASS-PRD.md. Surface: /strata/compass.

6. The surfaces

Six interactive surfaces plus the essay hub, all built from the same excavation, all hydrated the same way (Section 7).

The Collection ↔ Graveyard are deliberate twins (settled vs. abandoned). The Organization is the reflexive surface — the org chart measured by its own work, the public companion to essay No. 10 ("The Dig Itself"), and the place every team's live count converges. Its "planned" row is now empty: all eight teams report a real number.

7. Architecture & the hydration contract

The runtime constraint: a Vercel serverless route cannot spawn git over full history. So Strata splits cleanly into generation (on a runner with the full clone) and serving (static, from committed data).

1. Generation. run-strata.mjs orchestrates the agents in dependency order (Excavation → Lab → Games → Curator → Cartography → Atlas → Compass last, since it reads the others' outputs). Each writes a JSON/CSV artifact to the git-ignored output/. build-report.mjs then distills the small, source-free runs/{digest,latest,org}.json and copies the surface data into runs/.
2. Committed data. Only the small strata/runs/* files are committed — never output/.
3. Build-time hydration. Each surface is a self-contained HTML file with a baked SAMPLE constant and a loader. At build, src/lib/strata/surface.js replaces that constant with the committed JSON (buildSurface(kind)), so the page renders real data on first paint with no client fetch, falling back to the baked sample only when the data is genuinely absent. The routes are force-static.

This is the single mechanism behind all six surfaces; adding a seventh is a config entry, a route, and a companion tile — nothing more.

8. The honesty discipline (the spine of the whole system)

Strata is a system whose only real risk is looking authoritative while being wrong. The entire design is bent against that. Five concrete commitments, each enforced in code:

1. Real numbers or none. Every published figure traces to a commit. The essays were re-authored against the actual run; "the numbers in these essays are real or they are nothing."
2. A headline self-measurement per team. The Lab reports residual_load; Atlas reports rationale_rate (72 %); Compass reports arcs-surfaced-vs-held (1 of 16); essay No. 10 reports the survival metric's own blind spots (37 of 71 areas exceed 100 % survival on relocated/generated content). Each team measures the thing that would most embarrass it.
3. Absence is data. A decision with no recorded rationale, an arc that doesn't clear the floor, a team with no shipped output — each is reported as exactly that, never fabricated. The org chart marks unbuilt teams "planned" with no count; Compass is allowed to surface nothing.
4. No model where a model would invent. Atlas's optional LLM only condenses text already in the record; Compass uses no model at all, because any model role there would be generating a prediction. Forecasting is the one forbidden act.
5. Declared, versioned parameters. The residual lexicon, the decision typology, the Compass floor and strength weights are explicit constants, versioned with their spec documents, so every result is reproducible and every threshold's effect is inspectable.

9. Automation & operations

Strata maintains itself. .github/workflows/strata-run.yml:

• Weekly + on-demand. A Monday cron, the /strata UI dispatch, and workflow_dispatch all trigger a full-history run on a runner with fetch-depth: 0. The agents need no secrets.
• Refresh-as-PR. The refreshed runs/* data lands via peter-evans/create-pull-request on bot/strata-refresh — a reviewable checkpoint that also permanently heals run-to-run data drift. A quiet week opens no PR.
• Autofix in CI. claude-agent.yml runs anthropics/claude-code-action (live API key), so an OWNER comment @claude … on any PR runs Claude in CI to push fixes.
• Partial-run honesty. Agents run with continue-on-error; the report is still built and flagged, artifacts still upload, and the job fails red — a broken run is diagnosable, never silently green, and never opens a refresh PR.

10. Metrics & success criteria

• Reproducibility. A re-run on the same commit reproduces every dataset exactly. Met.
• Groundedness. Every published Atlas rationale is a verbatim substring of its commit (audited: 0 fabrications); every Compass arc cites ≥ 3 dated signals. Met.
• Coverage honesty. Each team's self-measurement is reported, believable, and tunable. Met.
• The org tells the truth about itself. Tiles flip planned→built only on real output; the count is never fabricated. Met — all eight built, zero planned.
• Self-maintenance. Surfaces refresh weekly without human effort and surface a PR only on real change. Shipped.

11. Build status & roadmap

Shipped (the canonical run, 31 May 2026): all eight teams; the ten-essay series; six interactive surfaces + the org surface; the substrate schema; the weekly self-refresh automation. The "planned" row is empty.

Open / future bets (not committed):

• [F1] Bedrock persistence by default. Today the surfaces run source-free and database-free; persisting the normalized record to Supabase (--full) unlocks cross-run trend analysis.
• [F2] Atlas PR-body enrichment in CI. The richest rationale seam (PR descriptions) needs a token on the runner; wiring it raises rationale_rate and pr_enriched.
• [F3] Compass over time. Re-run Compass across snapshots to watch arcs form and dissolve — the longitudinal version of "what you keep circling." Gated on the signal staying honest.
• [F4] Cross-repository runs. The pipeline is parameter-logged and repo-agnostic; running it on several repositories turns a case study into a comparison — the Lab's stated paper direction.
• [F5] The same discipline for the other pipelines. Enrichment, CRSS, and Heddle could adopt the weekly-refresh-as-PR pattern; each has its own data and secrets, so it is separate work.

12. What not to touch / invariants

• Never publish a number that isn't real. This is the product, not a guideline.
• Never let an agent depend on git at request time. The generation/serving split is load-bearing.
• Never commit output/. Only the small, source-free runs/* files are committed.
• Compass never predicts. It is an evidence board; that boundary is not a future feature.
• Parameters change in their spec doc and their constant together, or the dataset stops being reproducible.

13. Appendix — source-of-truth map

So this document resists drift, here is where each claim actually lives:

Strata is, in the end, one idea applied eight ways: a repository remembers everything, including what it tried to forget, and a patient enough reader can give that memory back — provided the reader never claims to know more than the record does.