esrlabs · AmmarAbouZor · Mar 2, 2026 · Mar 2, 2026
diff --git a/.ai/INDEX.md b/.ai/INDEX.md
@@ -0,0 +1,21 @@
+# Chipmunk AI Context Index
+
+## Overview
+
+This is the canonical entrypoint for repository AI context.
+Read this file first at the start of each session, then follow linked indexes and context files.
+Context is split into two domains:
+
+- `.ai/knowledge/`: stable repository/module context
+- `.ai/playbooks/`: procedural recipes and runbooks
+
+## Start Here
+
+1. `.ai/INDEX.md` (this file)
+2. `.ai/knowledge/INDEX.md` (module mapping and dependency flows)
+3. `.ai/playbooks/INDEX.md` (playbook catalog and template)
+
+## Navigation
+
+- Module and target context: `.ai/knowledge/INDEX.md`
+- Procedures and execution recipes: `.ai/playbooks/INDEX.md`
diff --git a/.ai/knowledge/INDEX.md b/.ai/knowledge/INDEX.md
@@ -0,0 +1,56 @@
+# Chipmunk Knowledge Index
+
+## Overview
+
+This is the canonical mapping for repository module context.
+All module context files live under `.ai/knowledge/` with a layout mirroring repository paths.
+
+## Start Here
+
+1. `.ai/knowledge/INDEX.md` (this file)
+2. `.ai/knowledge/cli/development-cli/AGENTS.md` (`cargo chipmunk` orchestration)
+3. `.ai/knowledge/application/apps/indexer/AGENTS.md` (Rust core)
+4. `.ai/knowledge/application/platform/AGENTS.md` (shared TS contracts)
+5. `.ai/knowledge/application/holder/AGENTS.md` (desktop host flow)
+6. `.ai/knowledge/application/client/AGENTS.md` (desktop application client)
+
+## Canonical Mapping
+
+| Repo Path | Context File | Primary Responsibility | Depends On |
+| :--- | :--- | :--- | :--- |
+| `application/apps/indexer` | `.ai/knowledge/application/apps/indexer/AGENTS.md` | Rust backend parsing/ingestion/search core | `.ai/knowledge/application/apps/indexer/stypes/AGENTS.md` |
+| `application/apps/indexer/stypes` | `.ai/knowledge/application/apps/indexer/stypes/AGENTS.md` | Rust/TS shared type source of truth | `.ai/knowledge/application/apps/protocol/AGENTS.md`, `.ai/knowledge/application/platform/AGENTS.md` |
+| `application/apps/protocol` | `.ai/knowledge/application/apps/protocol/AGENTS.md` | WASM encode/decode bridge around `stypes` | `.ai/knowledge/application/apps/indexer/stypes/AGENTS.md` |
+| `application/apps/rustcore/rs-bindings` | `.ai/knowledge/application/apps/rustcore/rs-bindings/AGENTS.md` | Native Node addon boundary over Rust `session` | `.ai/knowledge/application/apps/indexer/AGENTS.md` |
+| `application/apps/rustcore/ts-bindings` | `.ai/knowledge/application/apps/rustcore/ts-bindings/AGENTS.md` | High-level TS API over native addon/protocol | `.ai/knowledge/application/apps/rustcore/rs-bindings/AGENTS.md`, `.ai/knowledge/application/apps/protocol/AGENTS.md` |
+| `application/apps/rustcore/wasm-bindings` | `.ai/knowledge/application/apps/rustcore/wasm-bindings/AGENTS.md` | Browser-focused WASM utilities | `.ai/knowledge/application/client/AGENTS.md` |
+| `application/platform` | `.ai/knowledge/application/platform/AGENTS.md` | Shared IPC/types/env/logging for holder/client | `.ai/knowledge/application/client/AGENTS.md`, `.ai/knowledge/application/holder/AGENTS.md` |
+| `application/holder` | `.ai/knowledge/application/holder/AGENTS.md` | Electron host lifecycle/windowing/IPC bridge | `.ai/knowledge/application/platform/AGENTS.md`, `.ai/knowledge/application/apps/rustcore/ts-bindings/AGENTS.md` |
+| `application/client` | `.ai/knowledge/application/client/AGENTS.md` | Angular UI and feature surfaces | `.ai/knowledge/application/platform/AGENTS.md`, `.ai/knowledge/application/apps/rustcore/ts-bindings/AGENTS.md` |
+| `cli/chipmunk-cli` | `.ai/knowledge/cli/chipmunk-cli/AGENTS.md` | User-facing parser/export CLI | `.ai/knowledge/application/apps/indexer/AGENTS.md` |
+| `cli/development-cli` | `.ai/knowledge/cli/development-cli/AGENTS.md` | Build/test/lint task orchestrator for all targets | all major target contexts |
+| `plugins` | `.ai/knowledge/plugins/AGENTS.md` | WASM plugin contracts and artifacts | `.ai/knowledge/application/apps/indexer/AGENTS.md` |
+
+## Cross-Module Dependency Paths
+
+- Data path: `application/apps/indexer` -> `application/apps/rustcore/rs-bindings` -> `application/apps/rustcore/ts-bindings` -> `application/holder` and `application/client`.
+- Shared type path: `application/apps/indexer/stypes` -> `application/apps/protocol` -> `application/apps/rustcore/ts-bindings` and `application/platform/types`.
+- Plugin path: `plugins/` contracts/components -> `application/apps/indexer/plugins_host` runtime execution.
+- Orchestration path: `cli/development-cli` drives builds/tests/lints across all targets.
+
+## Development Workflow
+
+Primary entry point:
+
+- Build: `cargo chipmunk build [TARGET] -u print`
+- Test: `cargo chipmunk test [TARGET] -u print`
+- Lint: `cargo chipmunk lint [TARGET] -u print`
+
+| Component | Target | Path |
+| :--- | :--- | :--- |
+| Rust Core | `core` | `application/apps/indexer` |
+| Angular UI | `client` | `application/client` |
+| Electron App | `app` | `application/holder` |
+| Shared TS | `shared` | `application/platform` |
+| Rust FFI | `binding` | `application/apps/rustcore/rs-bindings` |
+| TS Bridge | `wrapper` | `application/apps/rustcore/ts-bindings` |
diff --git a/.ai/knowledge/application/apps/indexer/AGENTS.md b/.ai/knowledge/application/apps/indexer/AGENTS.md
@@ -0,0 +1,81 @@
+# Indexer (Rust Core) Context
+
+## Overview
+
+Indexer is the Rust backend core for parsing, ingestion, indexing, search, merging, and plugin execution.
+
+## Start Here (First Files to Open)
+
+1. `application/apps/indexer/Cargo.toml`
+2. `application/apps/indexer/processor/src/lib.rs`
+3. `application/apps/indexer/processor/src/producer/`
+4. `application/apps/indexer/processor/src/search/`
+5. `application/apps/indexer/sources/src/`
+6. `application/apps/indexer/parsers/src/`
+7. `application/apps/indexer/session/src/`
+8. `application/apps/indexer/plugins_host/src/`
+9. `.ai/knowledge/application/apps/indexer/stypes/AGENTS.md`
+10. `application/apps/indexer/indexer_base/src/`
+
+## Architecture & Crates (Condensed)
+
+- `processor`: ingestion/search pipeline; `MessageProducer` coordinates `ByteSource` + `Parser`.
+- `sources`: byte ingestion from File/TCP/UDP/Serial/Process/Pcap.
+- `parsers`: DLT/SOME-IP/Text parsing.
+- `indexer_base`: shared chunk/time/progress primitives.
+- `session`: Rust API boundary consumed by Node bindings.
+- `stypes`: Rust/TS shared type source (`ts-rs` generation).
+- `plugins_host`: `wasmtime` runtime for parser/bytesource plugins.
+- `merging`: multi-source chronological merge logic.
+
+## If You Need X, Go to Y
+
+- Add or modify ingestion sources: `application/apps/indexer/sources/src/` + `application/apps/indexer/processor/src/producer/`.
+- Add or modify parser formats: `application/apps/indexer/parsers/src/` + `application/apps/indexer/processor/src/`.
+- Change search/filter behavior: `application/apps/indexer/processor/src/search/`.
+- Adjust random-access/file-windowing: `application/apps/indexer/processor/src/grabber/` + `application/apps/indexer/indexer_base/src/`.
+- Change Node-facing Rust API: `application/apps/indexer/session/src/` then `application/apps/rustcore/rs-bindings`.
+- Change plugin runtime/lifecycle: `application/apps/indexer/plugins_host/src/` + `plugins/`.
+- Add shared Rust/TS type: `application/apps/indexer/stypes` -> `application/apps/protocol` -> `application/apps/rustcore/ts-bindings` -> `application/platform/types`.
+- Investigate pipeline failures: start in `application/apps/indexer/processor/src/producer/`.
+
+## Cross-Module Dependency Map
+
+- Core path: `indexer` -> `application/apps/rustcore/rs-bindings` -> `application/apps/rustcore/ts-bindings` -> `application/holder` / `application/client`.
+- Shared type path: `application/apps/indexer/stypes` -> `application/apps/protocol` -> `application/platform/types`.
+- Plugin path: `plugins/` contracts/artifacts -> `application/apps/indexer/plugins_host`.
+
+## Landmarks and Hotspots
+
+- `MessageProducer` in `processor` (pipeline orchestration).
+- `PluginsManager` and plugin host wrappers in `plugins_host`.
+- `Session` APIs in `session` and FFI alignment with bindings.
+- `spawn_blocking` boundaries in parse/search-heavy paths.
+- `TimedLine` and chunk/progress types in `indexer_base`.
+- High-context boundaries:
+    - `session` -> `rs-bindings` -> `ts-bindings`
+    - `stypes` -> `protocol` -> `platform/types`
+    - `plugins_host` runtime contract alignment with `plugins/`
+
+## Generated Artifacts and Source of Truth
+
+- Shared Rust/TS messages:
+    - Source: `application/apps/indexer/stypes/src/`
+    - Generation: `application/apps/indexer/stypes/generate.sh`
+    - Output: `application/apps/indexer/stypes/bindings/`
+    - Consumer copy target: `application/platform/types/`
+
+## Development & Tech Stack
+
+- Target: `core`
+- Build: `cargo build`
+- Test: `cargo test`
+- Lint: `cargo clippy`
+- Async runtime: `tokio` (`spawn_blocking` for CPU-heavy work)
+- Errors: `anyhow` + `thiserror`
+- Safety: `undocumented_unsafe_blocks` denied
+
+## Testing
+
+- Snapshots: `insta` (set `CI=true` in CI runs)
+- Fuzz/property testing: `proptest`
diff --git a/.ai/knowledge/application/apps/indexer/stypes/AGENTS.md b/.ai/knowledge/application/apps/indexer/stypes/AGENTS.md
@@ -0,0 +1,49 @@
+# Shared Types (stypes) Context
+
+## Overview
+
+`stypes` is the source of truth for data structures shared between Rust core and TypeScript surfaces.
+
+## Start Here (First Files to Open)
+
+1. `application/apps/indexer/stypes/Cargo.toml`
+2. `application/apps/indexer/stypes/src/`
+3. `application/apps/indexer/stypes/generate.sh`
+4. `application/apps/indexer/stypes/bindings/`
+5. `application/apps/protocol/src/lib.rs`
+6. `application/apps/rustcore/ts-bindings/spec/session.protocol.spec.ts`
+7. `application/platform/types/`
+
+## If You Need X, Go to Y
+
+- Add or modify a shared type: `application/apps/indexer/stypes/src/`.
+- Control TS export shape: Rust derives/attributes (`TS`, `#[ts(export)]`) in `application/apps/indexer/stypes/src/`.
+- Regenerate TS bindings: `./generate.sh` in `stypes`.
+- Validate encode/decode compatibility: `application/apps/protocol` and `application/apps/rustcore/ts-bindings` protocol specs.
+- Update consumed TS contracts: `application/platform/types/`.
+
+## Cross-Module Dependency Map
+
+- Source of truth: `application/apps/indexer/stypes/src/`.
+- Consumed directly by Rust indexer crates.
+- Exported to TS via generated bindings and via `application/apps/protocol`.
+- Downstream consumers include `application/apps/rustcore/ts-bindings`, `application/holder`, `application/client`, and shared `application/platform/types`.
+
+## Landmarks and Hotspots
+
+- `TS` derivations and serde traits on core structs/enums.
+- `proptest` coverage for new types and serialization edges.
+- Type changes with optional/enum evolution are high-context because they affect protocol and frontend contracts.
+
+## Generated Artifacts and Source of Truth
+
+- Source: `application/apps/indexer/stypes/src/`
+- Generation: `application/apps/indexer/stypes/generate.sh`
+- Generated output: `application/apps/indexer/stypes/bindings/`
+- Consumer copy target: `application/platform/types/`
+
+## Tech Stack
+
+- `ts-rs` for TypeScript derivation.
+- `proptest` for property-based robustness.
+- `serde` for JSON/Bincode serialization.
diff --git a/.ai/knowledge/application/apps/protocol/AGENTS.md b/.ai/knowledge/application/apps/protocol/AGENTS.md
@@ -0,0 +1,44 @@
+# Protocol WASM Context
+
+## Overview
+
+`protocol` is the WASM encode/decode bridge around shared `stypes`, used by Node/Electron/TypeScript layers.
+
+## Start Here (First Files to Open)
+
+1. `application/apps/protocol/Cargo.toml`
+2. `application/apps/protocol/src/lib.rs`
+3. `application/apps/protocol/src/generate.rs`
+4. `application/apps/protocol/src/err.rs`
+5. `application/apps/protocol/test.sh`
+6. `.ai/knowledge/application/apps/indexer/stypes/AGENTS.md`
+7. `application/apps/rustcore/ts-bindings/spec/session.protocol.spec.ts`
+
+## If You Need X, Go to Y
+
+- Expose a new shared message type: `application/apps/protocol/src/lib.rs` macro invocations.
+- Change encode/decode wrapper generation: `application/apps/protocol/src/generate.rs` (`gen_encode_decode_fns!`).
+- Investigate WASM boundary errors: `application/apps/protocol/src/err.rs` and caller expectations in `application/apps/rustcore/ts-bindings`.
+- Validate protocol compatibility: run `cargo chipmunk test protocol -u print` and inspect TS protocol specs.
+
+## Cross-Module Dependency Map
+
+- Upstream schema source: `application/apps/indexer/stypes`.
+- Protocol output consumed by `application/apps/rustcore/ts-bindings`.
+- Indirectly feeds `application/holder` and `application/client` through TS bindings.
+
+## Landmarks and Hotspots
+
+- `gen_encode_decode_fns!` macro wiring in `application/apps/protocol/src/lib.rs`.
+- `serde-wasm-bindgen` conversion boundaries.
+- Type additions are high-context because they require synchronized updates across `stypes`, protocol, and TS tests.
+
+## Quick Commands
+
+- Build: `cargo chipmunk build protocol -u print`
+- Test: `cargo chipmunk test protocol -u print`
+- Lint: `cargo chipmunk lint protocol -u print`
+
+## Additional Notes (Not Default Workflow)
+
+- `application/apps/protocol/test.sh` can run property-test-heavy paths and may take significantly longer than typical target tests.
diff --git a/.ai/knowledge/application/apps/rustcore/rs-bindings/AGENTS.md b/.ai/knowledge/application/apps/rustcore/rs-bindings/AGENTS.md
@@ -0,0 +1,41 @@
+# Rust Bindings (FFI) Context
+
+## Overview
+
+`rs-bindings` is the native Node addon boundary between Rust core (`application/apps/indexer/session`) and Node/Electron.
+
+## Start Here (First Files to Open)
+
+1. `application/apps/rustcore/rs-bindings/Cargo.toml`
+2. `application/apps/rustcore/rs-bindings/src/lib.rs`
+3. `application/apps/rustcore/rs-bindings/src/js/session/`
+4. `application/apps/rustcore/rs-bindings/src/js/jobs/`
+5. `application/apps/rustcore/rs-bindings/src/js/converting/`
+6. `application/apps/indexer/session/src/`
+7. `application/apps/rustcore/ts-bindings/src/`
+
+## If You Need X, Go to Y
+
+- Add or modify session-facing operation: `application/apps/rustcore/rs-bindings/src/js/session/` plus matching `application/apps/indexer/session` API.
+- Add static job without active session: `application/apps/rustcore/rs-bindings/src/js/jobs/` (`UnboundJobs` path).
+- Debug Node-side lifecycle or callback bridging: `RustSession` flow in `application/apps/rustcore/rs-bindings/src/js/session/`.
+- Investigate Rust type translation to JS payloads: conversion paths in `application/apps/rustcore/rs-bindings/src/js/converting/`.
+
+## Cross-Module Dependency Map
+
+- Upstream Rust API comes from `application/apps/indexer/session`.
+- Exposes native addon consumed by `application/apps/rustcore/ts-bindings`.
+- Downstream UI surfaces are `application/holder` and `application/client` through TS wrappers.
+
+## Landmarks and Hotspots
+
+- `RustSession` lifecycle methods.
+- `UnboundJobs` APIs.
+- Runtime/thread boundaries (Tokio runtime interaction with Node threads).
+- Memory allocator configuration (`tikv-jemallocator`/`mimalloc`) is high-context and global-impact.
+
+## Development
+
+- Target: `binding`
+- Build: `cargo chipmunk build binding -u print`
+- Lint: `cargo clippy`
diff --git a/.ai/knowledge/application/apps/rustcore/ts-bindings/AGENTS.md b/.ai/knowledge/application/apps/rustcore/ts-bindings/AGENTS.md
@@ -0,0 +1,45 @@
+# TypeScript Bindings Context
+
+## Overview
+
+`ts-bindings` is the high-level TypeScript API over the native Rust addon, used by Electron holder and Angular client.
+
+## Start Here (First Files to Open)
+
+1. `application/apps/rustcore/ts-bindings/src/api/`
+2. `application/apps/rustcore/ts-bindings/src/api/executors/`
+3. `application/apps/rustcore/ts-bindings/src/native/`
+4. `application/apps/rustcore/ts-bindings/src/services/`
+5. `application/apps/rustcore/ts-bindings/spec/`
+6. `.ai/knowledge/application/apps/rustcore/rs-bindings/AGENTS.md`
+7. `.ai/knowledge/application/apps/protocol/AGENTS.md`
+
+## If You Need X, Go to Y
+
+- Change session flow (`grab`, `search`, filters): `application/apps/rustcore/ts-bindings/src/api/` and related executors.
+- Add or modify static utility jobs: `application/apps/rustcore/ts-bindings/src/services/` and provider/native wrappers.
+- Adjust progress reporting behavior: `application/apps/rustcore/ts-bindings/src/api/` and `application/apps/rustcore/ts-bindings/src/services/` tracker-related flow.
+- Debug binary protocol translation issues: `application/apps/rustcore/ts-bindings/src/native/` and `application/apps/rustcore/ts-bindings/spec/session.protocol.spec.ts`.
+- Validate end-to-end behavior with native addon: `application/apps/rustcore/ts-bindings/spec/` tests using Electron runtime.
+
+## Cross-Module Dependency Map
+
+- Consumes native APIs from `rs-bindings`.
+- Consumes protocol encoders/decoders from `application/apps/protocol`.
+- Provides API used by `application/holder` and `application/client`.
+
+## Landmarks and Hotspots
+
+- `Session`, `SessionStream`, `SessionSearch` classes.
+- `CancelablePromise` usage in long-running operations.
+- Executor classes for operation-specific orchestration.
+- Protocol and addon version drift is a high-context boundary.
+
+## Development & Testing
+
+- Target: `wrapper`
+- Build: `cargo chipmunk build wrapper -u print`
+- Test: `cargo chipmunk test wrapper -u print`
+- Lint: `cargo chipmunk lint wrapper -u print`
+- Benchmarks: `application/apps/rustcore/ts-bindings/spec/_session.benchmark.spec.ts`
+- Test defaults: `application/apps/rustcore/ts-bindings/spec/defaults.json`
diff --git a/.ai/knowledge/application/apps/rustcore/wasm-bindings/AGENTS.md b/.ai/knowledge/application/apps/rustcore/wasm-bindings/AGENTS.md
@@ -0,0 +1,40 @@
+# WASM Bindings Context
+
+## Overview
+
+`wasm-bindings` provides browser-facing Rust WASM utilities for fuzzy matching, ANSI rendering, and filter validation.
+
+## Start Here (First Files to Open)
+
+1. `application/apps/rustcore/wasm-bindings/Cargo.toml`
+2. `application/apps/rustcore/wasm-bindings/src/lib.rs`
+3. `application/apps/rustcore/wasm-bindings/src/`
+4. `application/apps/rustcore/wasm-bindings/spec/`
+5. `application/apps/rustcore/wasm-bindings/pkg/`
+6. `application/apps/rustcore/wasm-bindings/package.json`
+7. `.ai/knowledge/application/client/AGENTS.md`
+
+## If You Need X, Go to Y
+
+- Tune fuzzy matching output/scoring: matcher code in `application/apps/rustcore/wasm-bindings/src/`.
+- Change ANSI conversion or stripping behavior: ANSI conversion code in `application/apps/rustcore/wasm-bindings/src/`.
+- Update regex/filter validation messages: filter validation code in `application/apps/rustcore/wasm-bindings/src/`.
+- Validate browser compatibility regressions: WASM-facing specs in `application/apps/rustcore/wasm-bindings/spec/`.
+
+## Cross-Module Dependency Map
+
+- Built as a WASM package consumed by `application/client`.
+- Interacts with frontend behavior, not the Node native addon path.
+
+## Landmarks and Hotspots
+
+- `matcher` integration with `skim` scoring.
+- `ansi` HTML conversion rules and escaping.
+- `filter` regex validation and error messaging.
+- Browser runtime compatibility (Karma/ChromeHeadless) is the critical hotspot.
+
+## Quick Commands
+
+- Build: `cargo chipmunk build wasm -u print`
+- Test: `cargo chipmunk test wasm -u print`
+- Lint: `cargo chipmunk lint wasm -u print`