perf lab: add exact comparison twins and the MessagePipe semantic ledger

[wallstop]

Part of #500. Blocks #414. Depends on #506 for the canonical profile.

Question

For every DxMessaging-versus-MessagePipe row, can we measure the same topology and account for every semantic difference before interpreting the rate ratio?

Mechanism

A comparison can reward fewer handlers, different token topology, different payloads, fragmentation, or less lifecycle work. Without exact internal twins and a semantic ledger, the observed gap mixes implementation cost with contract differences.

Contract

Keep the public comparison idiomatic for both libraries. Benchmark-only normalized rows may remove one documented semantic cost at a time, but they never become production APIs or replace the public-contract result.

The ledger covers ordering, live activity, callback-time mutation visibility, snapshot timing, reset, reentrancy, exceptions, diagnostics, registration lifecycle, reclamation, thread ownership, and found/not-found behavior. Mark each dimension matched, extra, or absent.

Factors and workloads

• GlobalToOne, GlobalToMany, KeyedToOne, and the exact comparison struct.
• Filtered, PostProcess, and FilteredPostProcess.
• Canonical SubUnsub lifecycle.
• 16 tokens/handlers and 16 distinct targets with one hit.
• One token with four priorities.
• One interceptor, one post, and one interceptor plus one post.
• Targeted, broadcast, without-context, and global-accept-all siblings where applicable.
• MessagePipe physical-capacity fragmentation: one survivor at first, middle, and last slots; alternating holes; repeated churn while retaining one subscriber.
• Callback-time subscribe and dispose traces for both libraries.

Record handler count, token count, priorities, route cardinality, payload shape, live count, physical capacity, mutation, diagnostics, allocation work, and cleanup work.

Primary response

The fraction of public comparison rows with both a machine-verified exact internal topology twin and a complete semantic-ledger entry.

Independent unit

One comparison row for coverage. Performance confirmation uses independent clean IL2CPP build pairs; rows, launches, and cycles remain nested observations.

Effect threshold

Acceptance requires 100% twin and ledger coverage. Any unmatched dimension prevents a parity claim for that row. A movement beyond 3% is material but cannot be attributed until both validators pass.

RED proof

• First require the existing 16-token/16-handler internal row for GlobalToMany in ComparisonDispatchTopologyTests; the stale four-handler mapping must fail.
• Change a twin's token, handler, priority, payload, or route cardinality and prove validation fails.
• Delete a required ledger dimension and prove schema validation fails.
• Break callback fan-out or teardown reconciliation and prove exact assertions fail.
• Use a stale external package version or source hash and prove validation fails.

GREEN suites

Pass bridge tests, internal twins, exact fan-out assertions over warm-up plus all emitted operations, source/version pin validation, allocation and lifecycle checks for SubUnsub, renderer/schema tests, and the canonical Standalone comparison suite.

Stop rule

If a public-contract twin is impossible, document why and add a benchmark-only normalized lower bound. Do not manufacture equivalence, hide extra work, or use the normalized row alone to close #414.

Immutable evidence

Retain topology manifests, semantic ledger, MessagePipe version/source hashes, raw cycles, callback totals, live/physical capacity, allocation/lifecycle results, rendered tables, and public-versus-normalized mappings.

Dependencies

Timing conclusions also depend on #508 for content-addressed evidence and #510 for calibrated inference.

Completion checklist

• Correct the GlobalToMany mapping.
• Add every missing topology twin.
• Characterize MessagePipe fragmentation and callback-time mutation.
• Add the versioned semantic-ledger schema.
• Complete the ledger for every comparison row.
• Add deliberate topology and ledger drift tests.
• Retain public and normalized results separately.
• Link the completed ledger from Research MessagePipe performance targets and formal parity for 4.0.0 #414, perf: calibrate the IL2CPP measurement system and attribute dispatch cost #500, and PLAN.md.

[wallstop]

PR #524 corrects the two existing one-handler and sixteen-handler twins in f6ec701. GlobalToMany now maps to the actual sixteen-handler benchmark, the unused primary token is removed, and comparison setup explicitly disables diagnostics. Tests inspect actual topology, token metadata, all six registration counters, callback results, and cleanup.

Four actual-source metadata guards passed locally; disabling the twin flag produced the expected failure. The 12 live dispatch/cleanup cases compile but require native Unity execution. Exact-head CI is running.

Seven remaining comparison topologies and the semantic ledger remain open. Two mapped rows and metadata checks are not full parity or a new throughput claim.

[wallstop]

The latest comparison CI failure was a test counter error, not a subscriber-topology defect. RegisteredUntargeted counts message-type/priority buckets. Six 16-subscriber cases incorrectly expected 16 buckets, although those handlers correctly share one priority-zero bucket.

Commit dde3d52 in PR #524 expects that one bucket and adds exact registration-row cardinality. Exact token metadata, independent subscriber registrations, callback counts, twin topology, and cleanup checks remain enforced. An actual bridge probe observed one bucket, 16 tokens, 16 registrations, and 272 callbacks for 17 emissions.

The original fixture failed 300 of 600 executions; the correction passed 600/600 and an independent 600/600 rerun. Fresh Unity 6000.4.6f1 passes all 16 topology tests with zero skips. New-head CI is pending.

This verifies the two existing twins only. The seven missing comparison topologies and remaining semantic ledger are still open.

[wallstop]

PR #528 adds the seven missing exact public comparison topology twins in 3ff163d, alongside the existing GlobalToOne and GlobalToMany internal rows. Contracts cover actual registration topology, payload/priority/route dimensions, exact callback fan-out and cleanup. Deliberate drift changes one dimension while preserving cardinality; churn checks inspect real register/remove history, including a negative lifecycle guard.

The new seven timing windows use the separate opt-in PerfTopologyTwin category and do not extend ordinary PerfComparison CI. Contract checks remain short. The runbook and canonical comparison skill describe all nine mappings and retain the separate-process measurement requirement.

Compilation with real Unity 2021 references and metadata/category checks pass; native execution is pending in PR #528. Full semantic-ledger, fragmentation, callback-time MessagePipe characterization, package-source hashes and durable native results remain open. Nine mapped/tested topologies alone do not establish #414 parity.

[wallstop]

The fragmentation and callback-mutation investigation now includes actual runtime observations from the pinned Unity package sources, rather than source predictions alone.

Compiled all 47 MessagePipe 1.8.2 and 153 UniTask 2.5.11 C# source files unchanged against Unity reference assemblies, then executed the synchronous broker paths on .NET 9.0.16/linux-arm64. All 200 vendor files match the retained registry archives byte-for-byte. MessagePipe tag 1.8.2 resolves to commit 58516c36d4465a7b6396b7850a4ad7e03326998c; archive SHA-256 is 70c3879f8ce72e37ae453669763d9f1b3b2d4fe693c05135210892b159109830.

37 deterministic cases pass in each of two fresh processes. Both complete trace outputs have SHA-256 35b2e092d08be2575905e208ba2722e19a3f7ed999e13df4e28ab2a324840c65. Independent review verified source, probe, binary and result identities and found no remaining issues.

Observed for both keyless and keyed brokers:

• Adding B from callback A with spare array capacity invokes B during the same publish.
• Adding E from A when four slots are full grows the array; E waits for the next publish.
• Removing pending B without growth skips B immediately. Growing first and then removing B still invokes B from the outer publisher's older array.
• A sole survivor at physical slots 0, 8 or 15 retains capacity 16. Alternating holes, repeated churn, stale disposal, nested publish and exception recovery have recorded traces and actual slot counts.
• Removing the final subscription removes its keyed holder. Disposing the entire core retains its empty keyed holder, a separate behavior that the cleanup observations preserve.

The observed storage comes from the actual pinned broker/FreeList fields; no routing model substitutes for production. All cases reconcile subscription cleanup to zero. Execution or cleanup errors produce a failing process exit.

Scope: synchronous int payloads, default options, one managed thread. These are managed characterization results, not native Unity, allocation, throughput, or parity evidence. Raw files currently remain local under .artifacts/session263-messagepipe-managed/; they have not been published as an immutable bundle. The complete schema, checked-in ledger, source-drift gates, cross-library traces and native acceptance remain open.

[wallstop]

Local Unity 6000.4.6f1 verification is now available at PR #528 head 2037ef6. The combined PlayMode filter for MessagePipeSemanticCharacterizationTests, DifferentialBusTraceTests, and UntypedStructPayloadTests passes all 202 cases twice with zero failures/skips (0.645694 and 0.222503 seconds). This includes the 78 tracked MessagePipe characterization cases; no new timing windows run.

The loaded runtime and test DLLs were matched to retained symbols through actual module IDs, CodeView identities, and full PDB checksums. All 214 repository source checksums match, including 77 runtime files. Both runs returned to the original clean scene. This is local editor/Mono evidence, not IL2CPP or a cross-library parity claim. Raw results and identity proofs remain local; immutable retention and the complete semantic ledger remain open. The current-head standalone comparison job is still queued.

[wallstop]

Dependency scope clarification for the preceding local Unity result: the shared macOS editor uses MessagePipe 1.8.1, while CI and the verified managed-source test rig use 1.8.2. The local DLL/PDB identities verify, but 32 of its 47 MessagePipe document hashes differ from the 1.8.2 archive; BOM/line-ending variants do not explain them. The host package manifest confirms 1.8.1.

Thus the local 202-pass runs are compatibility observations on 1.8.1. They do not supply the missing native proof for the pinned 1.8.2 dependency. The 214 repository/runtime source matches remain valid. No shared-editor dependency was modified. The standalone comparison job for current head 2037ef6 is still queued.

[wallstop]

Final-head comparison-contract execution is verified at 6dab5d54 in run 33994132092.

The uploaded comparison-contracts-6000.5.2f1 artifact (9978362980) matches API SHA-256 ddcfa40a8863d9a3a8ccc3801718d3f64fd89dddbbf2488f9f573aa5367c987a. Independent NUnit XML parsing reports 363 passed, 33 skipped, and zero failed in 15.8942592 seconds. All 87 MessagePipeSemanticCharacterizationTests cases pass without skips. This includes four untouched-default diagnostics lifecycle cases, four nested middleware exception/recovery cases, and the registered-inactive DxMessaging owner case.

The retained Unity Package Manager log reports com.cysharp.messagepipe@1.8.2 in the actual generated CI host. This is Unity 6000.5.2f1 PlayMode evidence for the pinned package. It is separate from the earlier local macOS compatibility runs using 1.8.1. No DLL/PDB byte identity or IL2CPP execution is claimed for this contract artifact.

These cases improve the executable characterization evidence. They do not complete the full semantic ledger or establish performance parity. The standalone comparison benchmark job is still running, and the broader acceptance items remain open.

[wallstop]

PR #534 integrates the semantic ledger and declared-source verification into the existing comparison pipeline. The missing ledger previously let reports and reducers consume rows without a machine-checked account of their semantic and topology differences.

The shared JSON/schema covers nine rows and all 108 required classifications. Existing registry consumers, including extraction, rendering, and paired reduction, validate it. Negative cases reject each deleted dimension, 81 well-typed topology changes, deleted source records, and invalid shared source references. The data contains portable repository paths and pinned external sources, with explicit logical-test-only provenance and no invented executed/native result.

The existing Unity configurator and build modifier emit fresh package-resolution descriptors. The harness checks pinned MessagePipe version and all 33 declared source files before/after build, then binds that evidence to the player manifest and raw result/log hashes. Repository digest normalization and exact compiler-input digests are separate. Result/log digests are explicitly pre-redaction; they do not authenticate later sanitized artifact bytes. No Unity launch or timing window was added.

Independent review verified the 21 repository hashes, 12 MessagePipe source hashes, archive identity, and eight logical test references. The real PowerShell guard accepts the pinned bytes and rejects source mutation, missing/wrong-case files, stale version, duplicate resolution, and a well-shaped wrong digest. Local integrated script tests pass; native pinned-package build acceptance is pending in #534.

Keep #507 open. The ledger remains parity-ineligible. Full native semantic sibling experiments, unresolved physical/allocation details, any normalized measurements, and durable post-redaction evidence remain campaign work; shape validation and source identity do not prove semantic equivalence or parity.

[wallstop]

PR #534 now has accepted comparison evidence at commit a197afdf6298573da639eb168c81f90729cf3d73.

The corrected comparison job passed, and artifact 9981706070 uploaded successfully. Its ZIP SHA256 matches GitHub metadata. All 30 files across the Standalone and PlayMode contract archives pass the production sensitive-data scanner and serialized-redaction safety checks.

The uploaded source report contains all 33 unique catalog identities and matching paths/hashes: 21 repository sources, with portable and exact Windows compiler-input hashes checked separately, plus 12 exact MessagePipe sources. Fresh Unity resolution records MessagePipe 1.8.2. Ledger, catalog and schema digests match measured source. The report records a 643-file player manifest and one validated run.

Standalone passed 55 cases; PlayMode contracts passed 363, including all 87 semantic characterization and 63 topology cases. Each mode has 33 explicitly unsupported library/scenario skips and zero failures. All five canonical IL2CPP Release profile checks and the pinned CPU/process checks pass. From a clean checkout of the measured commit, the production validator reproduces all 48 supported comparison rows and the uploaded paired summary exactly.

The source report now stores opaque identifiers as scalar sourceRef values in an array. The previous sources.dxToken object correctly triggered credential-container rejection. Actual-producer regression coverage proves safe evidence survives unchanged while true sensitive containers still fail; no scanner rule was weakened.

This remains bounded source/build/run association. Pre-redaction XML hashes do not authenticate changed sanitized uploads. Player binaries are absent from the archive, so their hashes and the full compiled closure cannot be independently reverified there. These checks do not establish semantic parity. The ledger remains draft and parity-ineligible. Native semantic sibling experiments and durable campaign evidence remain open under #507/#508.

[wallstop]

The existing opt-in PerfTopologyTwin scope was executed at source 1590dd334982cd832e65401138b5bd11904d520b through the maintained local Unity bridge.

Unity 6000.4.6f1 Editor PlayMode Mono x64 Debug passed all seven sibling windows in 35.8249794 seconds with zero failures, skips, or inconclusives. KeyedToOne, PriorityOrdered, Filtered, PostProcess, FilteredPostProcess, and StructNoBox each measured zero managed allocation calls. SubUnsub retained 20,000 registration allocation calls and 3,920,000 informational bytes in its separate allocation probe. Run GUID: 755dfe38-fdcc-47a5-9bcb-bcd14ffaa0ec. Raw and cleanup identities match; passive cleanup restored the original clean saved scene with no framework errors.

The completion checklist now reflects the already-delivered mapping, nine topology twins, fragmentation/mutation characterization, versioned 108-entry ledger, and drift tests. This local Mono run adds execution evidence without expanding recurring CI. It is not Standalone IL2CPP evidence, a MessagePipe comparison, a confirmatory interval, or immutable retention. Public-versus-normalized durable results and the final completed-ledger link gate remain open.

[wallstop]

Published the historical session 247 paired public-comparison inputs and normalized decision separately as an immutable prerelease: https://github.com/Ambiguous-Interactive/DxMessaging/releases/tag/perf-evidence-session247-paired-screen-replay-r1-75a42003

The archive contains the original bracket declaration plus first/center/last raw summaries. The manifest separately carries the reducer output. A fresh remote clone at verifier 1590dd334982cd832e65401138b5bd11904d520b anonymously downloaded the asset, verified all hashes, and reproduced the original uninterpretable decision because Filtered and FilteredPostProcess sentinels exceed the 3% band. Archive SHA-256 is 1cf1acfef029c3d29a178e71f719fdb76b754283e5911ca034b8bb47592dd478; manifest SHA-256 is 75a42003e501936051ee7e80e4be25d1d1b9449a8a21bdade522c3451ae747d2.

The public-versus-normalized retention checklist item is now complete. The unfavorable verdict is preserved; this does not establish parity, semantic equivalence, or a performance improvement. The final completed-ledger link gate and broader native sibling work remain open.