Documentation conventions

DataVault-backed projects keep documentation on three orthogonal axes. This page is the canonical definition of which axis owns what — consumer repositories link here instead of writing their own regulations.

Axis 1 — Narrative (human)

The "why" of the project: research questions, hypotheses, per-experiment reasoning. Files live in the consumer repository and are hand-edited.

FileScopeWhat goes in it
note/RESEARCH_PLAN.mdmonorepoCross-project research questions (RQ1, RQ2, …). Problem setting, success criteria, which project owns each RQ.
projects/<P>/note/STATUS.mdper-projectRolling status, project-local hypotheses (reference RQs by number), TODO, known issues, update log.
projects/<P>/note/OBSERVABLES.mdper-projectThe measurement × reference convention: which quantities we measure, against which exact solution / ED / QAtlas reference, and the data_schema_version that produces them.
projects/<P>/experiments/EXP-NNN-<slug>/README.mdper-experimentPurpose / Hypothesis / Design / Log / Result summary / Conclusion / Next actions.

Rules:

  • RQs are defined in RESEARCH_PLAN.md and referenced elsewhere. Do not copy RQ definitions into STATUS.md.
  • OBSERVABLES.md is updated whenever the bench.jl schema (data_schema_version) changes.
  • EXP-NNN READMEs are the single narrative source per experiment — no separate docs/src/results/ tier.

Axis 2 — Provenance (DataVault, automatic)

The "what ran when" layer: config snapshots, git hashes, run progress, figures, schema. Entirely produced by DataVault; never hand-edited.

FileWritten byPurpose
out/data/<P>/<run>/README.mdbuild_experiment_reportPer-run machine report: identity, code versions, config summary, ledger progress, event timings, figure index, schema.
out/data/<P>/<run>/schema.tomlbuild_experiment_report (first-write-only)Writer package / version / dataschemaversion / introspected JLD2 keys. Never overwritten; new writer identities spill to schema.toml.vN.
out/data/<P>/INDEX.mdbuild_experiments_indexPer-run row: snapshot, completed count, parent git hash, latest activity.
out/data/<P>/<run>/config_snapshot.tomlVault constructor (first-write-only)Exact TOML the run was launched with.
out/data/<P>/<run>/ledger.csvbuild_ledgerPer-completed-key row with git hash, timestamp, tag value.
out/figure/<P>/<run>/figures.tomlarchive_figure!Append-only figure version chain with SHA-1 dedup.
out/.datavault/<P>/<run>.log.tomlVault constructorFrozen discovery anchor (log_toml_version contract).

Rules:

  • Every provenance file is idempotent. Re-running the compute + analyze scripts must not corrupt them.
  • Narrative files link to provenance; they never duplicate.
  • Deleting a file in out/ is permitted (data layer), but deleting anything under out/.datavault/ breaks discovery.

Axis 3 — Infrastructure (DataVault, fixed)

Shared workflow / TEMPLATE / regulations — owned by DataVault and not replicated per project.

File / APILocationRole
experiment_template()DataVaultReturns the canonical EXP-NNN/README.md template string.
new_experimentDataVaultScaffolds experiments/EXP-NNN-<slug>/README.md with auto-incrementing ID.
build_narrative_indexDataVaultParses front-matter across experiments/EXP-*/README.md into an aggregated markdown table.
docs/src/workflow.mdDataVault docsCanonical 4-stage process (Plan / Freeze / Run / Analyze).
docs/src/conventions.mdDataVault docsThis page.

Rules:

  • Consumer repos do not ship their own TEMPLATE/ or WORKFLOW.md. They link to the DataVault docs instead.
  • When the workflow evolves, it is updated here (DataVault's docs), merged, and consumer repos pick it up by bumping the submodule.

Why three axes?

Keeping the axes separate makes documentation sustainable:

  • Changing the physics (new observable, new RQ) touches only Narrative.
  • Re-running a compute touches only Provenance (the human text is untouched).
  • Updating the workflow itself is a DataVault release — consumer repos don't need to re-learn the process.

When writing a new file, ask: "which axis owns this?" If the answer requires more than one axis, split the file.