Skip to content

fix(workspaces): cap WorkspaceKernel in-memory file path to prevent DO resets#621

Draft
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-code/workspace-kernel-file-memory-cap
Draft

fix(workspaces): cap WorkspaceKernel in-memory file path to prevent DO resets#621
posthog[bot] wants to merge 1 commit into
mainfrom
posthog-code/workspace-kernel-file-memory-cap

Conversation

@posthog

@posthog posthog Bot commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Summary

The WorkspaceKernel Durable Object was resetting on the /workspaces/<id> file flow, throwing paired "isolate exceeded its memory limit" and "storage operation exceeded timeout" errors. The cause is in workspace-kernel-file-commands.ts: it materializes whole file bodies in the DO's ~128MB isolate (object.arrayBuffer() on upload, readFileBytes() on read) while interleaving SQL storage writes. Since the product allows single uploads up to 200MB, a large enough file blows the isolate memory ceiling or holds the storage output gate past its timeout, resetting the object mid-operation.

@cloudflare/shell (the ShellWorkspace backing storage) only exposes whole-buffer readFileBytes/writeFileBytes — there is no streaming API to route bodies through. So this bounds the in-memory path by size instead, which the report sanctions as the minimum fix. Oversized files now fail cleanly rather than resetting the object:

  • Upload — rejected above the kernel cap at the route (handled 413), and again defensively in the DO before buffering.
  • Read — refuses to load over-cap bodies (covers files ingested before the cap existed).
  • Preview — skips the memory-heavy pdfium/photon-wasm decode for large files (best-effort; degrades to no thumbnail).

Caps live in a new workspace-kernel-file-limits.ts with the rationale documented.

Why

Production error tracking flagged WorkspaceKernel DO resets on the core file flow. It's contained today (a single user/session/workspace) but the root cause scales with file size and usage, and a reset mid-operation risks losing concurrent in-flight writes. Note: files large enough to trip these caps already failed — just catastrophically, by resetting the whole object; they now fail with a handled error instead.

Test plan

  • pnpm run check (format, lint, types) passes
  • Upload a file above the kernel cap → clean 413, no DO reset
  • Upload a normal file → unchanged behavior, preview still generated
  • Large PDF/image (over preview cap, under content cap) → stored, preview skipped

Created with PostHog Code from an inbox report.


View with Codesmith Autofix with Codesmith
Need help on this PR? Tag /codesmith with what you need. Autofix is disabled.

Review in cubic

…O resets

The WorkspaceKernel Durable Object materializes whole file bodies in its
~128MB isolate on upload ingest and read (@cloudflare/shell only exposes
whole-buffer read/write, so there is no streaming path). A large enough file
exceeds the isolate memory limit or holds the storage output gate past its
timeout, resetting the object mid-operation and risking in-flight writes.

Bound the in-memory path by size so oversized files fail cleanly instead of
resetting the object:
- Reject uploads above the kernel cap at the route (handled 413) and again
  defensively in the DO before buffering.
- Refuse to load over-cap bodies on read (covers files ingested before the cap).
- Skip the memory-heavy preview decode (pdfium/photon-wasm) for large files.

Files large enough to trip these caps already failed today — just
catastrophically, by resetting the whole DO.

Generated-By: PostHog Code
Task-Id: a2948632-0f77-473d-ac45-0e43e79edfd2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

0 participants