perf lab: define cold, burst, and queued tail-latency protocols

[wallstop]

Part of #500. Supports #505. Depends on #506, #508, #509, and #510.

Question

Can we measure first-use and p50/p95/p99 delivery behavior without warm averages, coordinated omission, timer overhead, queue semantics, or correlated frame samples hiding user-visible latency?

Mechanism

A warmed five-second throughput window hides type initialization, first registration, first dispatch, snapshot creation, periodic bursts, queue delay, backlog, overflow, and rare stalls. The current minimum-of-seven lifecycle row is a diagnostic best-case floor, not a user latency distribution.

Contract

Keep three result families separate:

• Cold: no warm-up; time one operation with setup and teardown outside the sample, across fresh players/builds.
• Burst: apply a fixed open-loop arrival trace and measure completion of 1/4/16/64/128/256-message frame bursts.
• Queued: measure enqueue-to-final-handler completion and report producer, wait/scheduler, and drain time separately.

Use preallocated timestamp storage and a monotonic clock. Keep profiler recorders, memory snapshots, and development instrumentation out of verdict windows. Declare ordering, frame phase, capacity, offered load, overflow, backpressure, cancellation, disposal, and drain budget.

Factors and workloads

• Process-cold fresh starts and randomized equivalent type-cold shapes.
• First registration, dispatch, plan build, trim, and teardown.
• Idle, moderate, saturation, and overload open-loop arrival traces.
• Immediate main thread, same-frame queue, and next-frame pipeline.
• Capacity, producer/worker count, batch size, drain budget, overflow, and handler-cost distributions.
• Ordered, cross-type, targeted hit/miss, and mixed-message holdouts.
• Timer-only, no-op, 10% cold-delay, and deterministic 1%-tail-stall controls.

Primary response

Each experiment registers one family-specific primary: median fresh-player cold latency, p99 burst completion, or p99 enqueue-to-handler latency at fixed offered load. Do not aggregate the families.

Independent unit

A fresh clean build/player start for process-cold work and a complete host-stable player session for burst/queue work. Frames and messages within a session are correlated.

Effect threshold

Protocol validation must detect the 10% cold delay and 1%-frequency tail delay while identical controls pass calibrated equivalence. Later ordinary candidates use #510; alternative queued tiers retain #505's 20% promotion threshold.

RED proof

• Add first-operation-only work and prove only cold moves.
• Stall exactly 1% of bursts and prove p99 moves while the mean can remain misleading.
• Inject drop, duplicate, reorder, overflow, starvation, and cancellation faults and prove test: build a differential bus state-machine oracle #509 detects them.
• Measure timer-only/no-op controls and reject a protocol whose observer cost is material.

GREEN suites

Pass protocol/reducer and exact-quantile fixtures, timer monotonicity/overhead, open-loop offered-load checks, differential-oracle scenarios, allocation rows, LeakWatcher, lifecycle/reentrancy, capacity/trim checks, and Standalone IL2CPP execution.

Stop rule

Report p99 as N/A when sample size, effective independence, or interval precision is inadequate. Do not publish when timer overhead is material, offered load is not fixed, controls fail, coordinated omission remains, or queue semantics are incomplete. Do not call repeated same-type operations cold.

Immutable evidence

Retain raw cold trials, arrival traces, per-burst or sampled-message timestamps, clock metadata, achieved/offered load, depth/age/overflow traces, counts, player/profile hashes, quantile and interval analysis, and minimized semantic failures.

Dependencies

This protocol gates latency claims and any #505 architecture promotion.

Completion checklist

• Specify process-cold and type-cold protocols.
• Specify open-loop burst and queue boundaries.
• Define quantiles, intervals, effective sample rules, and N/A criteria.
• Calibrate clock and timestamp overhead.
• Pass cold-only and 1%-tail controls.
• Pass queue ordering, overflow, and lifecycle fault tests.
• Run in Standalone IL2CPP.
• Publish replayable raw timing evidence.