Let --print-hashes baseline draft rows so locking a goldenset row is one edit

[swinney]

Objective

Let an operator compute a row's source_hashes before declaring it locked, so locking a golden set row is one edit instead of a two-step that leaves the row authoritative but unbaselined in between.

Context you need

The drift subcommand landed in e3bbb55ff0bf36ccad3c90ebadbc0bcb777c6799 (PR #144, group 4 of the openspec change maintain-ragas-goldenset). It re-fetches every locked row's sources, hashes the extracted text, and compares against the row's stored source_hashes.

The tool never writes the bank. --print-hashes emits a paste-ready source_hashes block and a human pastes it in. That is the only path by which a baseline is ever recorded.

The defect: baselines are only available for rows that are already locked.

• src/utils/goldenset_maintenance.py:1572 — find_drift skips any row where row_status(record) != "locked", so a draft row's sources are never fetched or hashed.
• scripts/benchmarking/goldenset_maintenance.py:855 — _print_hashes iterates report.rows, which by the above contains only locked rows.
• scripts/benchmarking/goldenset_maintenance.py:898-903 — when nothing is emitted, the CLI explicitly instructs: "set status: locked on the row, then re-run this to get its block."

So the only supported workflow is: mark the row locked → run drift --print-hashes → paste. Between steps 1 and 3 the row is authoritative with no baseline. A crash, a forgotten re-run, or an interrupted session leaves it that way, and drift then reports it under "no baseline recorded" until someone repairs it by hand.

Measured scale, run against examples/benchmarking/fasrc_ragas_queries.json at e3bbb55f:

{'locked': 0, 'draft': 105, 'total': 105,
 'anchor_type': {'easy_retrieve': 29, 'reasoning': 73, 'should_refuse': 3}}

Every row in the bank is draft. Confirming the bank therefore means 105 passes through the two-step, each with a window where the row is locked and unbaselined.

Why this was deferred rather than fixed in #144: the finding came from a post-merge adversarial review whose recommendation was to add "one explicit lock operation that ... atomically writes status: locked together with source_hashes". That requires the tool to write the bank file, which task 4.4 of the openspec change forbids ("confirm no detection path writes to the bank file") and which the change's proposal-only design (D3) rejects outright. The objective above is the design-consistent form of the same fix: make the hash available earlier so the human's single edit can set both fields at once.

Constraints

• Branch from origin/dev. Open the PR with gh pr create --repo fasrc/archi --base dev. Never PR to upstream / archi-physics/archi. Never commit to dev directly.
• No Co-Authored-By or session trailers on commits. Commit messages: short, lowercase.
• TDD: write the failing test first, watch it fail, then the minimum code to pass.
• Commit only green. The pre-commit hook runs scripts/gate.sh (format → lint → test, ≥80% diff coverage on changed lines). Never --no-verify.
• Prefix every python/pytest/openspec command with the conda env: export PATH=/home/austin/miniforge3/envs/archi/bin:$PATH.
• The tool must not write the bank file. This is load-bearing, not a preference. Any design in which the tool edits status or source_hashes is out of scope and will be rejected.
• Update docs/docs/benchmarking.md in the same PR (repo rule: user-facing CLI changes ship their docs).

Plan

Single PR; this is one behavior change with its docs.

1. RED test in tests/unit/test_goldenset_maintenance.py: find_drift(..., baseline_drafts=True) (or your chosen parameter name) fetches and hashes a draft row's sources and exposes a fresh digest for them.

2. RED test for the invariant that makes this safe — and get this right, it is the whole risk of the change:

   Task 4.2 of the openspec change says "draft rows ... are never drift-checked, regardless of hash." A draft row pulled in for baselining must therefore never:

   • appear in report.drifted,
   • appear in report.unbaselined (it is not a finding that a draft has no baseline — that is its normal state),
   • trigger an LLM call,
   • count toward checked_rows,
   • affect the abstention decision.

   It contributes a hash block and nothing else. Assert each of these.

3. Implement: plumb the flag through find_drift in src/utils/goldenset_maintenance.py. Prefer a representation that makes the invariant structural rather than a set of conditionals a later change can drift from — e.g. carry baseline-only rows in a separate field on DriftReport rather than mixing them into rows.

4. RED test + implement in tests/unit/test_goldenset_maintenance_script.py: a CLI flag (suggested --baseline-drafts, usable with --print-hashes) emits a block for a draft row. Assert the bank file is byte-unchanged by the run.

5. Replace the "lock first" message at scripts/benchmarking/goldenset_maintenance.py:898-903 — it currently teaches the unsafe order. Point at the new flag instead.

6. Docs: update the "Recording a baseline" section of docs/docs/benchmarking.md so the documented workflow is a single edit (compute the hash, then paste status: locked and source_hashes together). Every drift example in that file must carry the required --allowed-hosts flag — check all of them, this exact drift has bitten the file before.

Commands

export PATH=/home/austin/miniforge3/envs/archi/bin:$PATH

# baseline the current state before you start
python -m pytest tests/unit/test_goldenset_maintenance.py \
                 tests/unit/test_goldenset_maintenance_script.py -q   # expect: 273 passed

# during work
python -m pytest tests/unit/ -q
black src/ scripts/ tests/ -q && isort src/ scripts/ tests/ -q
openspec validate maintain-ragas-goldenset --strict
(cd docs && mkdocs build --strict)

# every documented drift invocation must carry the required allowlist
grep -c 'goldenset_maintenance.py drift' docs/docs/benchmarking.md
grep -A 3 'goldenset_maintenance.py drift' docs/docs/benchmarking.md | grep -c 'allowed-hosts'

Acceptance criteria

• python -m pytest tests/unit/ -q passes; total count > 1290.
• The gate passes at commit time with ≥80% diff coverage; no --no-verify in the branch history.
• A test asserts a draft row supplied for baselining produces a fresh digest.
• Separate tests assert that same draft row does not appear in drifted, does not appear in unbaselined, does not trigger an LLM call, and does not change checked_rows or abstained.
• A CLI test asserts the bank file is byte-identical before and after a --print-hashes run that includes drafts.
• grep -n 'set .status: locked. on the row, then re-run' scripts/benchmarking/goldenset_maintenance.py returns nothing.
• The two grep counts in Commands above are equal (every documented drift example carries --allowed-hosts).
• openspec validate maintain-ragas-goldenset --strict passes.
• cd docs && mkdocs build --strict exits 0.

Start here

export PATH=/home/austin/miniforge3/envs/archi/bin:$PATH
git fetch origin && git switch -c fix/goldenset-baseline-drafts origin/dev
sed -n '1540,1620p' src/utils/goldenset_maintenance.py     # find_drift's locked gate
sed -n '855,905p' scripts/benchmarking/goldenset_maintenance.py  # _print_hashes + the message to replace

Then write the failing test from step 1.

Originating review: post-merge adversarial review of PR #144.