Skip to content

feat(file-explorer): lazy per-directory watchers + deferred teardown#112

Merged
den-sq merged 1 commit into
ChengLabResearch:mainfrom
tavateva:lazy-file-explorer
Jul 7, 2026
Merged

feat(file-explorer): lazy per-directory watchers + deferred teardown#112
den-sq merged 1 commit into
ChengLabResearch:mainfrom
tavateva:lazy-file-explorer

Conversation

@tavateva

@tavateva tavateva commented Jul 7, 2026

Copy link
Copy Markdown

Closes #110. Refs #102.

Summary

Replaces the single recursive Watcher(root, {recursive:true, depth:6, limit})
in src/main/event-handlers/filesystem.ts with a Map<path, WatcherEntry> of
non-recursive watchers. fetch-folder-contents(root) closes every open
watcher, resets the aggregate visible-add counter, and starts one watcher on
the root - root is now just a distinguished expanded folder. Two new IPC
handlers drive lazy loading:

  • expand-folder(subPath) starts a non-recursive watcher on subPath (or
    cancels a pending teardown if one is queued, making open-close-open
    idempotent).
  • collapse-folder(subPath) schedules teardown of the subPath watcher after
    COLLAPSE_TEARDOWN_DELAY_MS (30_000 ms). Cancelled by a subsequent
    expand-folder(subPath). Root is exempt; collapse-of-root is a no-op.

FILE_EXPLORER_WATCH_DEPTH is removed. FILE_EXPLORER_WATCH_LIMIT is kept
but its role changes from a per-watcher watcher-package limit to an
aggregate session budget summed across every open watcher; a watcher's
contribution is reclaimed on teardown.

src/shared/constants.ts is a new dependency-free module holding
COLLAPSE_TEARDOWN_DELAY_MS so both the main process and the renderer import
the same value. tsconfig.node.json and tsconfig.web.json include it;
electron.vite.config.ts gets an @shared alias.

Renderer:

  • DirectoryContext exposes expandFolder(subPath) and collapseFolder(subPath).
    Collapse mirrors the main-side timing with a state prune scheduled via
    pruneSubtree(subPath); re-expanding within the window cancels both timers.
    Switching the root clears all pending timers.
  • DraggableEntry dispatches expand/collapse alongside its existing visual
    isCollapsed toggle. Idempotent - main-side handlers no-op when state
    already matches. The chevron now renders on every folder (not just
    non-empty ones) so an unopened lazy folder is expandable.
  • The nodes: {} IFrame broadcast is unchanged; no plugin-facing API in
    this PR (that lands in Plugin-facing directory request API with recursive semantics (part of #102) #111).

src/preload/index.ts documents the new IPC channels; the electron-toolkit
preload accepts arbitrary channel strings so no per-channel type is required.

Acceptance criteria (from #110)

  • Opening a 100k-file folder produces state and events only for root's
    direct entries. (Root is watched non-recursively; batches only include
    root's direct children until subfolders are expanded.)
  • Expanding a subfolder streams that folder's direct entries only, no
    recursion. (New non-recursive watcher per expanded folder.)
  • Collapsing a subfolder drops its state after the delay unless
    re-expanded first. (Both main-side watcher teardown and renderer-side
    pruneSubtree are scheduled on COLLAPSE_TEARDOWN_DELAY_MS and are
    cancelled by a subsequent expand.)
  • File selection, drag-drop between folders, and new-folder uniqueness
    (FileExplorer.tsx:80 isUsed) continue to work. (No changes to the
    NodeChildren shape; handleFSEvent still populates the nested map from
    batch events, only lazily.)
  • npm run typecheck passes. (Verified locally; see below.)
  • Documented in documentation/docs/index.md (Large Folder Behavior)
    and documentation/docs/reference/technical-constants.md
    (COLLAPSE_TEARDOWN_DELAY_MS added; FILE_EXPLORER_WATCH_DEPTH removed;
    FILE_EXPLORER_WATCH_LIMIT role clarified).

Design notes / minor divergences

  • Chevron visibility change. The pre-existing DraggableEntry hid the
    chevron for empty folders (!isEmpty). Under lazy loading a folder starts
    with children: {} until first expand, so !isEmpty would hide the chevron
    on every not-yet-expanded folder and there would be no way to expand it.
    The chevron now shows on every folder regardless. True empty folders are
    still visually distinguished by the emptyFolder style.
  • Main-window teardown listener. handleEvents runs before
    createWindow, so getMainWindow() returns undefined at
    handler-registration time. The window closed listener is bound lazily on
    the first fetch-folder-contents call and rebinds if the window is
    recreated (e.g. macOS reactivation).
  • Per-watcher addCount also tracks unlinks. When an unlink /
    unlinkDir event fires, both the per-watcher and aggregate counters
    decrement. This keeps the aggregate budget honest across long-lived
    sessions where files come and go without triggering false limit warnings.
  • pruneSubtree reducer. Renderer-side prune walks the nested nodes
    map by absolute path equality (not by root-relative path decomposition)
    because FileSystemNode.path is already absolute; this keeps the reducer
    independent of the current root and makes it work uniformly for any
    subPath the main process might collapse.

Blockers / pre-existing issues

  • npm run lint fails with ESLint couldn't find an eslint.config.(js|mjs|cjs) file.
    This is the pre-existing ESLint 9 config drift called out in PR chore(constants): named ports w/ env override, tighten iframe allowlist, split memory interval, env-configurable shm_size #109; not
    introduced by this PR and not addressed here.
  • Manual smoke via npm run dev was not attempted; the automation host has
    no reachable display. A reviewer can run npm run dev, open a mid-size
    folder, and expand/collapse a subfolder to visually confirm:
    1. Initial open shows only root's direct entries in the panel.
    2. Clicking the chevron on a subfolder loads its direct children.
    3. Collapsing then re-expanding within ~30 seconds is instant with no
      visible re-fetch flicker.
    4. Collapsing and leaving collapsed for >30 seconds, then re-expanding,
      shows a fresh load.

Verification commands

npm ci                # completed
npm run typecheck     # passes (both tsconfig.node.json and tsconfig.web.json)
npm run lint          # fails with pre-existing ESLint 9 config drift; see above

Files changed

  • src/main/event-handlers/filesystem.ts - watcher-manager rewrite +
    expand-folder / collapse-folder handlers.
  • src/renderer/src/contexts/DirectoryContext.tsx - new
    expandFolder / collapseFolder methods, pruneSubtree reducer,
    timer bookkeeping.
  • src/renderer/src/components/MenuPanel/components/FileExplorer/components/DraggableEntry/DraggableEntry.tsx -
    dispatch expand/collapse on chevron click; always show chevron on folders.
  • src/shared/constants.ts - new; COLLAPSE_TEARDOWN_DELAY_MS.
  • src/preload/index.ts - documenting comment for the new IPC channels.
  • tsconfig.node.json, tsconfig.web.json, electron.vite.config.ts -
    include src/shared/**/* and register the @shared alias.
  • documentation/docs/index.md - Large Folder Behavior rewritten for the
    lazy model.
  • documentation/docs/reference/technical-constants.md -
    FILE_EXPLORER_WATCH_DEPTH removed, FILE_EXPLORER_WATCH_LIMIT role
    clarified, COLLAPSE_TEARDOWN_DELAY_MS documented.

…loses ChengLabResearch#110)

Replace the single recursive `Watcher(root, {recursive:true, depth:6, limit})`
in `src/main/event-handlers/filesystem.ts` with a `Map<path, WatcherEntry>` of
non-recursive watchers. `fetch-folder-contents(root)` closes all watchers,
resets the aggregate visible-add counter, and starts a single watcher on the
root - root is now just a distinguished expanded folder. Two new IPC handlers
drive lazy loading:

- `expand-folder(subPath)` starts a non-recursive watcher on `subPath`, or
  cancels a pending teardown if one is queued (idempotent open-close-open).
- `collapse-folder(subPath)` schedules teardown of the subPath watcher after
  `COLLAPSE_TEARDOWN_DELAY_MS` (30_000 ms). Cancelled by a subsequent
  `expand-folder(subPath)`. Root is exempt - collapse-of-root is a no-op.

`FILE_EXPLORER_WATCH_DEPTH` is removed; the lazy model makes it redundant.
`FILE_EXPLORER_WATCH_LIMIT` is retained but its role changes from a
per-watcher `watcher`-package limit to an aggregate session budget summed
across every open watcher (each watcher's contribution is reclaimed on
teardown).

`src/shared/constants.ts` is a new dependency-free module that hosts
`COLLAPSE_TEARDOWN_DELAY_MS` so both the main process and the renderer import
the same value. `tsconfig.node.json` and `tsconfig.web.json` include it;
`electron.vite.config.ts` adds an `@shared` alias for future reach.

Renderer changes:
- `DirectoryContext` exposes `expandFolder(subPath)` and `collapseFolder(subPath)`.
  Collapse mirrors the main-side timing with a state prune scheduled via
  `pruneSubtree(subPath)`; re-expanding within the window cancels both timers.
  Switching the root clears all pending timers.
- `DraggableEntry` dispatches expand/collapse alongside its existing visual
  `isCollapsed` toggle. Idempotent - main-side handlers no-op when state
  already matches. The chevron now shows on every folder (not just non-empty
  ones) so an unopened lazy folder is expandable.
- The `nodes: {}` IFrame broadcast is unchanged; no plugin-facing API in this
  PR (that lands in ChengLabResearch#111).

`src/preload/index.ts` documents the new IPC channels; the electron-toolkit
preload accepts arbitrary channel strings so no per-channel type is required.

Docs:
- `documentation/docs/index.md`: Large Folder Behavior rewritten for the lazy
  model.
- `documentation/docs/reference/technical-constants.md`: `FILE_EXPLORER_WATCH_DEPTH`
  removed, `FILE_EXPLORER_WATCH_LIMIT` role clarified, `COLLAPSE_TEARDOWN_DELAY_MS`
  added with source, value, rationale, behavior at teardown, and when to change.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
tavateva pushed a commit to tavateva/ouroboros that referenced this pull request Jul 7, 2026
…ntics (closes ChengLabResearch#111)

Adds `request-directory-contents` / `send-directory-contents-response`
message pair to the plugin iframe surface, so plugins can walk the
current directory root on demand after ChengLabResearch#106 stopped broadcasting the
full recursive tree.

Renderer:
- New valibot schemas in `src/renderer/src/schemas/iframe-message-schema.ts`:
  `RequestDirectoryContentsSchema` (plugin -> provider) and
  `SendDirectoryContentsResponseSchema` (provider -> requesting iframe).
  A recursive `PluginFileSystemNodeSchema` (via `lazy`) mirrors the
  renderer's `NodeChildren` shape so plugins get a drop-in replacement
  for the pre-ChengLabResearch#106 broadcast payload.
- `IFrameContext.tsx` gains `handleRequestDirectoryContentsRequest`
  wired alongside the existing `read-file`/`save-file`/`register-plugin`
  handlers. It reads the current root through a ref so the closure sees
  the latest `directoryPath` regardless of when the listener effect ran,
  applies a path-traversal guard (rejects `..` escapes and Windows
  drive-letter changes), and posts the response only to the requesting
  iframe. Root broadcast (`send-directory-contents { nodes: {} }`) is
  unchanged.
- Errors map to `denied` (traversal / no root / malformed),
  `not-found` (ENOENT / ENOTDIR), `limit` (enumeration truncated), or
  `internal` (anything else).

Main:
- `src/main/event-handlers/filesystem.ts` gains a `get-folder-contents`
  IPC handler backed by a top-level `getFolderContents` that does a
  one-shot `readdir` walk. Recursion is depth-first and bounded by the
  same `FILE_EXPLORER_WATCH_LIMIT` used by the watcher path; hitting
  the cap returns `{ nodes, truncated: true }` with the partial result.
- Existing watcher factory refactored to use a shared
  `shouldIgnorePathAgainstRoot` helper so both the watcher and the
  one-shot walk apply the same dotfile + `IGNORED_PATH_SEGMENTS` rule.
- Handler does not touch the watcher-manager state; enumeration is
  independent of expand/collapse.

Docs:
- `documentation/docs/guide/plugins.md` gets a "Requesting Directory
  Contents" section with request/response examples, the error-code
  taxonomy, and notes on the traversal guard, truncation semantics, and
  ignore-rule inheritance.
- `documentation/docs/reference/technical-constants.md` extends the
  `FILE_EXPLORER_WATCH_LIMIT` entry with its second role as the
  per-request enumeration cap, and points the "Plugin directory
  broadcast" section at the new on-demand API.

`src/preload/index.ts` documents the new IPC channel alongside the
existing `expand-folder` / `collapse-folder` entries.

Verification: `npm run typecheck` passes (both node and web tsconfigs).
`npm run lint` still fails with the pre-existing ESLint 9 config drift
called out in ChengLabResearch#109/ChengLabResearch#112, not introduced by this PR.

Path-traversal in the existing `read-file` / `save-file` handlers is
unrelated to ChengLabResearch#111 and left as a follow-up.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@tavateva
tavateva force-pushed the lazy-file-explorer branch from 15555f2 to c3f2d9d Compare July 7, 2026 13:59
@den-sq
den-sq merged commit 03c5d44 into ChengLabResearch:main Jul 7, 2026
1 check passed
@tavateva
tavateva deleted the lazy-file-explorer branch July 7, 2026 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Lazy file explorer watchers and lazy renderer state (part of #102)

2 participants