docs: funding proposals and work packages by cgeorgii · Pull Request #353 · tweag/cardano-hoarding-node

cgeorgii · 2026-03-26T17:22:08Z

Summary

Work packages for four funding proposals.

Reviewers: the main content to review is the work package READMEs linked below — one per proposal. The other files in the diff are design documents and structural reorganisation; they don't need detailed review at this stage. Follow the README links for nicely rendered views.

Work Packages

Proposal	Work Package
Embedded Consensus Validation of Blocks and Headers	README.md
Distributed Mode	README.md
Live Network Deployment	README.md
Transaction Collection	README.md

Full index: proposal/README.md

Notes

Each work package covers: high-level description, core objectives, expected value, metrics for success, milestones with acceptance criteria, and budget
Supporting design documents are retained alongside each work package for reference
Proposals not part of this cycle or unfinished are moved to proposal/deferred/

- `distributed-mode.md`: collector/coordinator role split, protocols, graceful degradation, and single-node vs distributed deployment - `mux.md`: typed protocol channels over ouroboros-network-framework, with single-node (queues bearer) and distributed (TCP) interpreters - `keyed-pubsub.md`: routing key extension to Pub/Sub for intra-collector event routing; migration path from the current broadcast model - `infrastructure.md`: AWS + NixOS + deploy-rs + sops-nix + sqitch stack, CI/CD pipeline, and what is needed from the infrastructure team

- Rename `infrastructure.md` → `infrastructure-base.md` to scope it to the single-node base stack

- `header-validation.md`: proposal — motivation, per-peer state concept, `InvalidHeaderReceived` event, forecast horizon behaviour, and open questions; scoped to proposal level (no implementation detail) - `header-validation-hoard-design.md`: implementation design — `HeaderStateHistory` background, new `ChainDB` effect operations (`GetHeaderStateHistory`, `GetLedgerForecast`), `HeaderValidation` component listeners, and `Main.hs` wiring - `embedded-consensus-hoard-design.md`: add Stage 1b entry and link to the new design doc; note it is independent of Stages 2–4

Each proposal area gets its own directory. Sub-proposals sit inside their parent. Standard filenames within each folder: - `proposal.md` — the proposal - `design.md` — the implementation design (where applicable) New structure: embedded-consensus/ proposal.md, design.md header-validation/ proposal.md, design.md distributed-mode/ proposal.md, mux.md, keyed-pubsub.md infrastructure/ overview.md, base-stack.md All cross-references updated to reflect new relative paths.

- Rename `embedded-consensus` → `validation`; flatten `header-validation` sub-folder into the parent - Move `maintenance` to a top-level proposal; add DoS prevention note - Add placeholder proposals: `transaction-collection`, `stretch/data-export`, `stretch/ui` - Remove empty `peer-management-api` directory (implemented)

- Add `docs/retrospective.md` summarising Cycle 1 milestone delivery, beyond-scope work, and adversarial behaviour detected in the wild - Add `proposal/infrastructure/proposal.md` with budget ask, AWS cost estimate (~$206/month, 6-month total ~$1,236), success criteria, and deliverables (NixOS module, OCI image, reference deployment) - Add NixOS module and OCI image sections to `infrastructure/base-stack.md` - Flesh out `proposal/validation/proposal.md`: rename from Embedded Consensus, add motivation, deliverables, success criteria, dependencies, and effort estimate (~1 FTE for 2 months) - Flesh out `proposal/transaction-collection/proposal.md`: explain the TxSubmission protocol constraint and dependency on infrastructure - Add motivation section to `proposal/distributed-mode/proposal.md` (Cardano Foundation user interview) - Add `proposal/block-header-relaying/proposal.md` placeholder

…ucture - Add `distributed-mode/work-package.md`: high-level description, core objectives, expected value, metrics, milestones (3), and budget - Add `validation/work-package.md`: same structure across 5 milestones (embedded ChainDB, block validation, header validation, API exposure, remove external node dependency) - Add `infrastructure/work-package.md`: deployment pipeline, live preprod deployment, and external operator artifacts milestones; cloud cost estimate and labour budget - Add `work-package-template.md` as a shared reference for all proposals - Update `distributed-mode/proposal.md`: add deliverables, success criteria, and block retention open question - Rename `infrastructure/base-stack.md` → `single-node.md` - Remove `keyed-pubsub.md` (superseded by ouroboros-network-framework)

- Rename proposal folders to match work package titles: `infrastructure/` → `live-network-deployment/` `validation/` → `embedded-consensus-validation/` - Rename `work-package.md` → `README.md` in each proposal folder - Move deferred/placeholder proposals into `deferred/` - Add `proposal/README.md` as top-level index linking all work packages - Fix internal links broken by moves

prednaz · 2026-03-27T02:08:51Z

please do not forget about #346 (comment).

docs/retrospective.md

proposal/embedded-consensus-validation/header-validation-design.md

proposal/embedded-consensus-validation/proposal.md

prednaz · 2026-03-27T03:04:03Z

"canonical chain" is not a well defined term, is it? i do not see it used in the network specification, haddocks, nor docs.cardano.org. i am not sure if you use it for the immutable chain or the chain selected by our personal local ChainDB or full node.

prednaz · 2026-03-27T03:06:41Z

ChainDB, and mini protocols are quite inconsistently sometimes formatted as code and sometimes not. not sure if it is worth harmonizing this.

proposal/embedded-consensus-validation/README.md

proposal/embedded-consensus-validation/proposal.md

proposal/embedded-consensus-validation/README.md

prednaz · 2026-03-30T00:29:40Z

the absence of mainnet plans might look a little peculiar.

- **retrospective.md**: correct TxSubmission deferral rationale — promoted outbound connections are also a valid path per network spec §1.4.1, though unproven in practice; clarify both paths - **embedded-consensus-validation/README.md**: lead with the attack defense motivation; add header envelope as a distinct invalidity class; tighten requirements list (stake distribution derives election, add `HeaderStateHistory` for KES); fix three-stage taxonomy in Core Objectives; define "canonical" as part of the immutable chain; add mainnet follow-on note; harmonise `ChainDB`/`ChainSync` code formatting throughout - **embedded-consensus-validation/proposal.md**: rename "state machine" to `HeaderStateHistory`; split header validation row into ledger forecast and `HeaderStateHistory` lifecycle subtasks - **embedded-consensus-validation/design.md**: correct `BlockRolledBack` comment (rollback ≠ orphaned); fix orphan classification — use `blockSealedClassifier` as the single trigger for both canonical and orphaned, dropping the incorrect `blockRolledBackClassifier`; add `withDB`/`closeDB` resource management; document that ChainDB manages fork selection internally with no external rollback API; confirm `BlockRejected` fires for invalid blocks on all candidate forks - **embedded-consensus-validation/header-validation-design.md**: use `getPastLedger` at intersection point instead of `getCurrentLedger`; thread intersection point through per-peer state and effect signature; fix `rewind Nothing` — treat as protocol violation, not normal path; expand beyond-horizon open question with explanation of why it occurs during initial sync; add attack mitigation open question for invalid/unvalidatable headers - **distributed-mode/README.md**, **live-network-deployment/README.md**: add Strategic Alignment sections

cgeorgii · 2026-03-30T09:16:09Z

@prednaz I addressed all your comments (and the ones in PR #346) in this commit.

Promotes the deferred transaction collection stub into a full work package, incorporating the duplex connection promotion approach discovered since the original deferral. - Remove the thin stub from `proposal/deferred/transaction-collection/` - Add `proposal/transaction-collection/README.md` — full work package covering the duplex promotion path, its current unvalidated status, the inbound connection fallback, and three milestones (2 + 3 + 1 weeks, 2 FTE) - Add `proposal/transaction-collection/proposal.md` — implementation guide with the investigation approach for diagnosing why no connections were promoted against preprod

Incorporate insights from a parallel proposal on the peer governor promotion mechanics: - **README.md**: explain the cold → warm → hot promotion chain as the likely reason no connections were promoted against preprod; note that the silent `ChainSync` stub may cause peers to treat us as uninteresting; expand Milestone 2 deliverables to include a minimal `ChainSync` server if the stub proves insufficient - **proposal.md**: add "Promotion path" section detailing the three promotion steps and why each may stall; expand investigation to cover the promotion chain and `ChainSync` interestingness; add caveats for peer churn and loopback risk

proposal/live-network-deployment/README.md

proposal/transaction-collection/README.md

Incorporate points raised in the team Slack discussion: - **README.md**: surface `LocalTxMonitor` as a third fallback path alongside inbound connections; add explicit coverage caveat (txs can only arrive from peers that choose to run TxSubmission toward us) - **proposal.md**: add coverage caveat section; expand ChainSync "interesting" concern with the specific `MsgAwaitReply`/stale-header disconnect risk; add `LocalTxMonitor` alternative section describing the privileged NodeToClient path and its tradeoffs

prednaz

thanks!

cgeorgii added 8 commits March 25, 2026 09:19

docs: scope infrastructure proposal to single-node

118b743

- Rename `infrastructure.md` → `infrastructure-base.md` to scope it to the single-node base stack