Skip to content

Latest commit

 

History

History
455 lines (389 loc) · 25.4 KB

File metadata and controls

455 lines (389 loc) · 25.4 KB

Changelog

All notable changes to tanstack-durable-object-sync are documented here.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning. While pre-1.0, the public API may change between 0.x releases.

[Unreleased]

[0.6.0] — 2026-07-27

Added

  • Typed oversize-frame handling (ADR-0018; part of #28). An oversize client mutation used to be dropped silently server-side (ADR-0012's maxFrameBytes guard), surfacing only as a confirmation timeout — and in production Cloudflare's ~1 MiB edge cap on inbound WebSocket messages means the frame may never reach the DO at all. The transport now guards before sending: a mut/call whose encoded size exceeds the 1 MiB edge cap rejects immediately with MutationRejectedError (code "FRAME_TOO_LARGE"), so the optimistic overlay rolls back promptly. The server's silent drop stays as defense in depth. Outbound gains a warn-only fixed 1 MiB threshold: a larger encoded frame logs a console.warn with size and collection but is still sent whole — column projection (#28a) remains the real fix for oversized full-row re-sends. The limits are infrastructure facts (Cloudflare's edge cap), not application preferences, so they are constants, not options.

  • Transport reconnect policy (ADR-0016; fixes #25, #26). One option, reconnectDelay?: number | ((attempt, closeCode?, closeReason?) => number | null) (null = stop), replaces reconnectDelayMs (breaking, pre-1.0: a number keeps the old meaning — the default policy's base delay). The default (defaultReconnectDelay) replaces the fixed interval with capped exponential backoff + full jitter (base 250 ms; cap 30 s; attempt counter resets on a successful open) and treats application close codes 4000-4999 as terminal, so an accept-then-close auth rejection (e.g. 4403) no longer retries forever. A terminal stop surfaces through the new onClosed(code, reason) hook, making auth closes distinguishable from transient drops.

  • Cohosting docs moved to recipes/cohosting.md and re-grounded. The README's cohosting section shrinks to the pitch, the code sample, and a pointer; the recipe carries the rules, an honest account of what is verified and how (CI fake-host tests vs. source audit at pinned versions vs. the untested wake restore), and a reframed @cloudflare/actors story. Verified against @cloudflare/actors@0.0.1-beta.6: the Actor class hibernates fine on its own but won't cohost out of the box (its Sockets helper takes over socket connections completely and Actor claims the DO-wide auto-response slot), while the Actors helpers (Alarms, Storage) compose cleanly with sync over a plain DurableObject base — which Actors' own examples document. Supersedes the 0.5.0 "host defect" phrasing.

Fixed

  • Subscriptions survive hibernation wake (ADR-0019; field report against 0.5.1). Hibernatable sockets survive a DO eviction by design, but the subscription registry was instance memory: a wake restored the socket set and nothing else, so an idle client's live queries went silently dead on a still-open socket — its own mutations still confirmed while deltas fanned out to nobody, and the client's only re-subscribe trigger (the close path) never fired. Subscriptions are now written through to a durable _sync_subs table keyed by a per-socket id tag stamped at accept, and restored onto surviving sockets during registerSync on every wake — before anything can dispatch or drain. No wire or client change: an unmodified 0.5.1 client against a fixed server recovers fully. Orphaned rows (a socket that dies without webSocketClose) are swept during the existing compaction housekeeping; no idle timers (ADR-0006 invariant intact). Pinned by tests/hibernation.test.ts under real evictions (evictDurableObject, unlocked by the vitest 4 migration — the eviction-based wake test issue #29 asked for), in five shapes: eviction after successful broadcasts, eviction before the first-ever broadcast, eviction of a cohosted base (tagged-restore branch, host socket untouched), a restored sub whose collection left the schema (reconciled: reset + row dropped, healthy subs untouched), and a restored predicate that no longer compiles (socket closed with a non-terminal code so reconnect re-subscribes — a reset would strand the query, adversarial review). Sockets accepted by a pre-fix build that survive an in-place upgrade carry no id tag and keep the old behavior (dead until reconnect) — a one-release sharp edge, documented in ADR-0019.
  • Rejection code now survives tx-dedup replay (#21). The dedup record persisted only the rejection message, so a client retrying the same txId got the reason with no machine-readable code — breaking code-based error handling on exactly the retry path it exists for. _sync_seen_tx gains an error_code column (added in place on wake for already-deployed DOs), and the replayed rejected frame is now shaped identically to the original: { code, message } when a code was recorded, { message } otherwise.
  • BLOB columns no longer corrupt to {} over the wire (ADR-0017, #27). workerd's SqlStorage returns BLOB values as bare ArrayBuffer, which the msgpack encoder fell through to encodeMap on (and the JSON debug codec stringified as {}) — clients silently received an empty object, for snapshots and deltas alike. Both codecs now normalize ArrayBuffer at emission, so BLOB columns arrive on the client as a Uint8Array with the exact bytes.

[0.5.1] — 2026-07-03

Fixed

  • registerSync now rejects tables with no usable internal rowid (ADR-0015). The cold-snapshot/fetch reader defaults to ORDER BY rowid when the client sends no orderBy. A WITHOUT ROWID table has no rowid, so that read threw no such column: rowid and hung the subscriber; a table with a declared rowid column shadows the internal one, so the read would silently sort by that arbitrary column. assertSyncCompatible now rejects both loudly at registerSync, alongside the existing INTEGER PRIMARY KEY guard. Ordinary rowid tables (the documented id TEXT PRIMARY KEY pattern) are unaffected.
  • SyncMixin's declared webSocketClose/webSocketError now match their implementation. Both were typed as returning Promise<void> while the mixin's overrides return void synchronously (there is nothing to await — both just drop bookkeeping for the closed socket). Retyped to void | Promise<void>, mirroring @cloudflare/workers-types's own DurableObject interface. webSocketMessage is genuinely async and is unaffected. Cosmetic: no runtime behavior changed, only the emitted .d.ts.

[0.5.0] — 2026-07-02

Added

  • Syncable(Base) mixin — cohost sync on any Durable Object base (ADR-0015). The sync machinery is now a curried mixin factory, Syncable<Env, TUser>()(Base), so one DO can be both its framework's host (the Agents SDK Agent, @cloudflare/think's Think, a bare DurableObject) and a tddc sync source — no dedicated sync DO, no mirror write. Exposed from the root and a ./server/mixin subpath. Sync sockets carry a reserved tag and a plain attachment and claim only the /_sync path; all other traffic delegates to the host base, so the two protocols never cross (proof: partyserver's __pk filtering). Actor (@cloudflare/actors) is documented as unsupported because its Sockets helper adopts foreign sockets on wake. See the README "Cohosting" section.

  • Optional Standard Schema validation (ADR-0014). A collection's insert.schema (the row schema, which also infers the collection's Row) and update.schema (a partial patch schema), and a command's schema, are checked at runtime and rejected loudly on failure. Any ~standard library works (zod, valibot, arktype) and the framework adds no validator dependency. It is a gate, not a parser: the original value flows to handlers, so schemas must not rely on transforms, defaults, or coercion. See recipes/zod-standard-schema-collections.md.

Changed

  • SyncDurableObject is now Syncable()(DurableObject) — zero API change. All existing extends SyncDurableObject<Env, Claims> code, including this.sql, this.registerSync, this.runSyncedWrite, and an overridable parseAttachment, keeps compiling and behaving identically to 0.4.0 (the two DO-global side effects — ping/pong auto-response and PRAGMA case_sensitive_like = ON — stay ON for this base; they default OFF over any other base, opt in with this.sync.configure). The internal sql getter was removed from the mixin because it shadowed the host's sql tagged-template method; reach this.ctx.storage.sql directly on a non-DurableObject base.

  • Rejection reasons are surfaced uniformly for mutations and commands (revises ADR-0012 D3). An authorize throw or a schema validation failure now reaches the client with its reason (validation failures carry a VALIDATION code); only execute errors stay sanitized. Previously a command's authorize error was sanitized like its execute, unlike a mutation's.

[0.4.0] — 2026-07-01

Changed

  • Authoring is now an object schema, not a builder (ADR-0014). new SyncRegistry().defineCollection().defineMutation().defineCommand() is replaced by defineSync<User, Env>(), which binds identity/env once and returns { collection, command, schema }. Mutations are a closed insert/update/delete trio co-located on the collection (mirroring @tanstack/db's onInsert/onUpdate/onDelete — a custom mutation type is now structurally unrepresentable), superseding ADR-0001 D11 and absorbing ADR-0010's row manifest: the row type lives on the collection (sync.collection<Message>({ pk })) instead of a third SyncRegistry generic. The DO registers the schema value with this.registerSync(schema). Breaking, no compat shim.

Added

  • Typed commands, end to end (ADR-0014). export type Api = typeof schema is the whole client contract: new WebSocketTransport<Api>() exposes a typed transport.call.<command>(args) proxy plus a typed low-level sendCall(name, args) (txId generated internally via crypto.randomUUID()), and doCollectionOptions<Api, "table">({ … }) infers the row type from the schema — one source of truth across server and client.
  • examples/multi-do — a two-Durable-Object example: one transport per DO, a React SyncProvider/useSync keyed by DO so command namespaces never collide, and a client-side cross-DO feed.

[0.3.3] — 2026-06-13

Fixed

  • A filtered subscription's membership no longer depends on which path decided it (ADR-0013). The SQL snapshot and the JS delta/catch-up evaluators disagreed on two operators, so two clients could see different rows for the same where depending on connection timing:
    • ne crashed the delta path and hung the client. The SQL floor accepted ne but @tanstack/db's evaluator has no ne (not-equal is not(eq(...))); its compile error escaped handleSub uncaught, so no reset was sent. ne is now off the floor and rejected with reset like any unsupported operator, and a defensive guard turns any predicate-compile failure into a reset rather than a hang. Use not(eq(...)) for not-equal (unchanged for real clients).
    • like was case-insensitive in the snapshot but case-sensitive in deltas. The DO now sets PRAGMA case_sensitive_like = ON, making SQLite LIKE match @tanstack/db's case-sensitive like on every path. (#17)

Changed

  • Operator floor for server-side filtering dropped ne — it is now eq, gt, gte, lt, lte, like, in, and, or, not, exactly the set the SQL and JS evaluators agree on row-for-row. like is now case-sensitive. (#17)

[0.3.2] — 2026-06-13

Added

  • Wire-input hardening at the server boundary (ADR-0012). Frame-shape guards drop malformed frames — the socket survives and answers the next valid frame — and inbound limits bound frame size and per-connection subscription count. (#15)

Fixed

  • A failed initial connect no longer wedges the client transport. A WebSocket that never open()s fires no close event, so the auto-reconnect couldn't run and the cached rejected connectPromise wedged the transport permanently — every later connect() returned the same rejection. On a failed open the transport now clears the promise and re-arms the reconnect while subscriptions are live. (#12)

Changed

  • Reconnect catch-up no longer issues an N+1. Delta hydration batches keyed reads in chunks (≤64) and the per-table changelog read uses the (tbl, seq) composite index, so a 500-key catch-up issues ~8 queries instead of 500. Behavior is unchanged. (#14)

Security

  • Mutation execute errors are sanitized. A failed execute now returns a generic "mutation failed" rather than leaking SQLite/internal detail to the client; authorize errors still pass through. (#15)

Internal

  • Server error-path and wire-level test coverage expanded (#11); dead snapshotAll removed and CDC trigger DDL identifiers quoted (#13).

[0.3.1] — 2026-06-11

Fixed

  • Catch-up reinsert no longer wedges the client. A key deleted-and- reinserted while a client was away arrives in the catch-up as op=insert for a key the client still holds; TanStack's sync write throws DuplicateKeySyncError on that, aborting the whole catch-up transaction. The adapter now applies a held-key insert as the upsert it semantically is (the move-in update-upsert contract, ADR-0002 C4).
  • Cursor barrier (C1′). A snapshot or catch-up served on a socket with still-buffered coalesced deltas could advance the client's cursor past an undelivered write (multi-collection reconnect; drop before the tick lost the write). The server now flushes the socket's pending deltas before any synchronous cursor-advancing emission — ADR-0002 C1 generalized from committed to all cursor boundaries.
  • Reconnect window no longer kills subscriptions. The reconnecting flag was set inside the reconnect timer, so a mutation fired within the reconnect delay of a drop established the fresh socket before the timer ran — with no resubscribe, leaving every subscription silently dead on the new socket (and the late timer wedged the flag). The flag is now set when the reconnect is scheduled, so whichever connect wins — timer- or demand-driven — resubscribes from the cursor.

[0.3.0] — 2026-06-09

Added

  • Typed mutations (ADR-0010). SyncRegistry takes a third generic — a collection-row manifest — so handlers are fully typed without casts: new SyncRegistry<TUser, Env, { messages: Message }>() types pk (must be a column) and each handler's op.cols per op (insertMessage, updatePartial<Message>, delete→none). Purely type-level; the untyped two-generic form still works (op.cols falls back to unknown).
  • Changelog time-based retention (ADR-0009). A new changelogRetentionMs knob (default 2 days) prunes _sync_changes rows older than the window, so the log is bounded by age, not just key-cardinality. A client reconnecting from beyond the surviving floor now receives a reset + full snapshot instead of an incremental delta. Set changelogRetentionMs: null to disable retention (compaction-only, the prior behavior).

Changed

  • Renamed RegistrySyncRegistry. Breaking, no compat shim. The old name was too generic and clashy; the public surface is uniformly Sync* (SyncDurableObject, runSyncedWrite, registerSync). Update your import and new SyncRegistry(...).
  • registerSync now reconciles CDC triggers to the registry instead of only adding them: triggers for a collection you've removed from the registry are dropped on the next registerSync, so an orphaned table stops firing capture triggers into _sync_changes. Trigger reap only — existing change rows are left untouched. (ADR-0008)

[0.2.0] — 2026-05-31

Changed

  • Author-owned schema; registerSync wires the sync. Breaking. Collection definitions are now { table, pk }ddl is removed. You create your tables yourself (raw DDL, Drizzle, any migrator), then call this.registerSync(registry) — typically in your constructor's blockConcurrencyWhile, after migrating. It validates each table (one PRAGMA: the pk is a sole TEXT client key — D9) and installs the CDC triggers. Lazy initRegistry-on-fetch is gone; schema exists before the first event. Forget the call → this.registry throws loud. This also retires runSyncedWrite's "caller ensures init" caveat. See ADR-0007.

  • The cursor fetch frame now mirrors @tanstack/db's LoadSubsetOptions. Breaking wire change (client and server ship together). The frame carries a base where plus a raw cursor: { whereFrom, whereCurrent } — TanStack's own CursorExpressions names and semantics (the cursor expressions exclude the base where) — replacing the previous private where/ties fields that combined the predicates client-side. The server now composes base AND whereCurrent (ties, unbounded) and base AND whereFrom (next page, bounded by limit). Behaviour is unchanged (identical compiled SQL); the win is traceability to upstream. A malformed cursor (a missing half) is now rejected loudly instead of degrading to an unbounded scan. See ADR-0005.

  • Collection pk validation accepts TEXT affinity, not the literal "TEXT". registerSync now admits any TEXT-affinity pk (TEXT, VARCHAR, CHAR, NVARCHAR, …) so ORM/migrator-generated DDL composes; INTEGER (rowid alias) and other non-TEXT affinities are still rejected loudly (they'd break optimistic id parity). Part of ADR-0007.

Added

  • runSyncedWrite(fn) — a protected SyncDurableObject primitive for server-originated writes (an agent inserting a row, a webhook, a cron/ alarm job, an admin edit, a bulk seed): apply a raw synchronous SQL closure in a transaction, then broadcast the resulting CDC to connected clients. Outside the client mutation flow — no txId, no receipt, no dedup (idempotency rides the collection's mandated stable keys). See ADR-0006.
  • examples/board — an at-scale stress example: 5,000 tasks on one DO, bounded window load, useLiveInfiniteQuery cursor scroll-back, and a mutable order key (updated_at) so voting/starring bumps a task to the top (move-in via the always-emit upsert). A server-side firehose makes the deferred bounded-window-under-churn limitation visible: loaded climbs past window.
  • A real-tie integration test (unbounded boundary ties + limited next page + base predicate composed into both halves) covering the cursor fetch against the DO.
  • A move-in integration test: a cold row bumped server-side arrives via the no-where live sub and upserts into the collection (ADR-0002 C4).

[0.1.0] — 2026-05-30

Added

  • afterCommit post-commit hook + env in handler context. Mutations gain an optional afterCommit(ctx) that runs fire-and-forget via waitUntil after the commit and receipt — the sanctioned home for external side effects a synchronous execute can't do (delete an R2 object, enqueue a job). It's isolated (a throw is logged, never affects the committed mutation) and owns its own idempotency. Handler contexts (authorize/execute/afterCommit and command execute) now receive the DO's env, typed via a new Env generic on Registry that defaults to unknown (existing Registry<TUser> unchanged). See ADR-0004.
  • Bounded initial load for on-demand windows. A live query's orderBy and limit are forwarded to the server so the initial snapshot is the bounded window (e.g. the most recent N rows) rather than the whole where subset. The live subscription's predicate stays the where clause, so entering rows are still delivered.
  • Cursor load-more (scroll-back). Extending a windowed query past its first page issues one one-shot paginated fetch (new fetch/page wire frames) carrying both halves of the cursor double-read — boundary ties (ties, unbounded) and the next page (where, bounded by limit), each combined with the base where. The server reads both at a single seq (atomic), and no new live subscription is taken — the window's deltas already flow over the existing where sub. See ADR-0003.

Fixed

  • Cursor load-more no longer resurrects a concurrently-deleted row. The earlier two-frame double request read the ties at one seq but applied them after a deferred merge; a live delete landing in between let the stale tie re-insert the deleted row, with no future delta to correct it. The double-read is now one atomic fetch, so the page applies in stream order before any later delta. See ADR-0003.
  • Cursor load-more no longer throws on overlapping boundary rows. Page rows are written insert-if-absent, so a boundary tie already in the window (or a row a concurrent live delta already refreshed) is skipped rather than re-inserted — which would otherwise throw DuplicateKeySyncError and abort the open sync transaction. The live where subscription stays the source of truth for rows currently in the collection.
  • Rejected subscriptions no longer hang preload(). A reset with no snap-end (the server's response to an unsupported predicate or unknown collection) now resolves the subset's load promise instead of leaving the live query waiting indefinitely.
  • On-demand orderBy IR shape. orderBy clauses from real live queries ({ expression, compareOptions }) were not recognised by the SQL compiler, so server-side ordering was silently dropped and the wrong rows were returned for a bounded window. The compiler now accepts the live-query clause shape.

[0.0.1]

Initial release — sync a TanStack DB collection to a Cloudflare Durable Object over a single WebSocket. Server-authoritative, single-writer, no CRDTs.

Added

  • Single-ordered-stream sync. Change-data-capture via SQLite triggers into one per-DO change log; one monotonic seq cursor drives live deltas, reconnect catch-up, and write confirmation. The client tracks a single appliedSeq — no second acknowledgement channel.
  • Optimistic mutations with single-stream confirmation (awaitSeq): mut (atomic row transactions) and call (named commands), exactly-once via txId dedup. Mutation execute runs inside transactionSync; authorize runs before it.
  • Client-supplied keys enforced at defineCollection (ULID/UUIDv7); the optimistic id equals the confirmed id.
  • Filtered subscriptions. A where predicate (a @tanstack/db BasicExpression) is evaluated server-side with @tanstack/db's own compiler, so operators match the client exactly; move-in/move-out handled without a before-image. Client-side write-outside-filter preflight (WriteOutsideSubError).
  • Subset shapingwhere/orderBy/limit/offset lowered into SQLite (operator floor: eq, ne, gt, gte, lt, lte, like, in, and, or, not); un-lowerable predicates are rejected, never silently full-scanned.
  • On-demand subsets (syncMode: 'on-demand') — load only the subsets your live-query where clauses request; each distinct where is one refcounted server subscription, released on the last unload. Writes outside every loaded subset are confirmed without stranding an optimistic row.
  • Egress coalescer — per-(sub, key) last-write-wins on a tunable tick, collapsing high-rate writes (e.g. streaming tokens) to one delta per tick; hibernation-native (no timer when idle).
  • Reconnect catch-up — auto-reconnect resubscribes from the applied cursor; the server serves a windowed delta or, past the retention floor, a reset + snapshot.
  • Compaction-defined retention — opportunistic (every N writes, off the response path via waitUntil; no idle-DO wakeups): the change log collapses to latest-op-per-key, with independent time-based dedup GC.
  • Multiplexing — many collections on one DO share a single WebSocket.
  • Client IVM — live queries (joins, filters, aggregates) run client-side via @tanstack/db; the DO stores and emits, never runs IVM.
  • Binary wire — MessagePack frame codec (JSON debug fallback) + a tagged value codec preserving bigint/Date/NaN/±Infinity/-0/undefined/Uint8Array.
  • Hibernating WebSocketsacceptWebSocket + auto-response ping/pong; per-socket identity via serializeAttachment.
  • Examplesexamples/chat (eager, multi-tab live sync) and examples/on-demand (on-demand subsets), both verified in a real browser.

Deferred (post-1.0)

  • Windowed pagination (orderBy/limit/cursor double-request with server-side window maintenance under churn).
  • isWhereSubset containment dedup of overlapping subsets.