feat(live): provision petition documents into predict inputs (#474)#534
Merged
Conversation
Closes #474. The input-richness half of the live-sources design — the step that moves cert predictions past base-rate calls. - Pipeline-side fetch + extraction, never agent-side: the live poller fetches the petition and the brief in opposition on the same distribution transition that queues prediction (record-complete moment; near filing time, since document links are a rolling ~5-Term window upstream). Idempotent per (kind, url) — a relist with unchanged filings costs nothing; a re-filed BIO supersedes. - Extraction: pypdf (born-digital filings under the e-filing mandate; a scanned paper filing degrades to empty text, never a crash), capped at live.document_text_cap (~40 dense pages) with truncation flagged. - QPLink is an outcome artifact, never an input — verified live: the /qp/ page is generated when certiorari is GRANTED and opens with the grant order (this also explains the probe's 1/64 QPLink coverage). The questions presented are derived from the petition PDF's own QP section instead, and QPLink joins the replay-redaction blocklist. - Storage discipline: extracted text lands in the corpus's new documents table (DVC, access-gated) — never the git ledger. provision-snapshot materializes it into the cell's gitignored record/documents/ with a documents.json manifest (metadata only), so cells read identical content with no fetch rights; an older remote blob reads as "no documents" on the ranged backend. - Prompt wiring: predict.md points agents at record/documents/ (anchor on the questions presented, weigh petition vs BIO, cite in reasoning.md), with the data-not-instructions guard extended to documents. Verified against the live site on the real Loper Bright docket: petition (49pp/74k chars), BIO (37pp/65k — the entry caption needed a 120-char window; amicus briefs excluded), and the derived questions presented (1.8k chars), none truncated, materialized through provision-snapshot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Resolves the CodeQL implicit-string-concatenation finding on #534: adjacent bytes literals inside the _pdf helper's object list are the classic missing-comma bug shape — a real miss there would silently merge two PDF objects. The intended concatenation is now an explicit `+`. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
CodeQL's implicit-string-concatenation finding is fixed in |
…nto dev/474-document-provisioning
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #474. The input-richness half of the live-sources design, on top of #472/#473.
Decisions the issue asked for
live.document_text_cap(~40 dense pages, truncation flagged in the manifest) — petition + BIO fetched, questions presented derived from the petition's own QP section (Rule 14.1(a) fronts it). Document metadata (URL, kind, entry date, pages) rides indocuments.json.documentstable (DVC-backed, access-gated — never the git ledger, respecting no-republication).provision-snapshotmaterializes text + manifest into the cell's gitignoredrecord/documents/, so every workflow already provisions documents with zero workflow changes and agents get the content with no fetch rights. A pre-documents remote blob reads as "no documents" on the ranged backend rather than failing the cell.Found by fetching for real
QPLinkleaks the outcome. The/qp/page is generated when certiorari is granted and literally opens with "THE PETITION … IS GRANTED … CERT. GRANTED 5/1/2023" — which also explains the probe's 1/64 QPLink coverage (the one granted case). It is never fetched, and it joins the replay-redaction blocklist.Verified
Real-site run on the Loper Bright docket: petition 49pp/74k chars, BIO 37pp/65k, derived QP 1.8k chars — none truncated — stored, then materialized via
provision-snapshot. Prompt wiring:predict.mdpoints agents atrecord/documents/(anchor on the QP, weigh petition vs BIO, cite inreasoning.md); the cert-prompt-tuning follow-up the issue mentions stays separate.Gate: ruff, mypy, pytest (812 passed, +15), export-schemas no drift. Out of scope, per the issue: circuit documents (RECAP Fetch/PACER).
🤖 Generated with Claude Code