Skip to content

refactor(core): consolidate semantic vector planning#1130

Merged
phernandez merged 2 commits into
mainfrom
codex-1110-vector-planner
Jul 21, 2026
Merged

refactor(core): consolidate semantic vector planning#1130
phernandez merged 2 commits into
mainfrom
codex-1110-vector-planner

Conversation

@phernandez

Copy link
Copy Markdown
Member

Why

The indexing pipeline had three separate places making semantic-vector planning decisions: embedding job planning, durable project-index resume planning, and oversized-entity shard planning. Keeping those decisions in separate modules made the single and batch paths harder to reason about and evolve consistently.

Part of #1110.

What Changed

  • Made EmbeddingIndexPlanner the canonical owner for target deduplication and fingerprints, batch-job chunking, durable resume plans, and per-entity shard plans.
  • Moved the project vector-sync source and execution entry point beside the canonical planner.
  • Removed the separate vector_sync_planning.py implementation.
  • Routed ProjectIndexRuntime and repository shard planning through the canonical planner.
  • Preserved the existing single-entity and batch execution entry points.

Implementation Details

  • Planner methods remain pure and queue-neutral.
  • Repository code passes only pending chunk keys into shard planning, keeping repository record shapes out of the indexing planner.
  • Existing project progress, logging, error, deferred-entity, and compatibility-constant behavior remains covered.
  • The benchmarked local and cloud embedding flush sizes are unchanged.

Testing

  • just fast-check
  • Focused planner/runtime/vector-sync suite: 38 passed; embedding_index_planning.py at 100% coverage
  • just fast-test: 78 passed, 1 skipped, 360 deselected
  • just package-check
  • just doctor
  • git diff --check

Risks

The main risk is an internal import or behavior drift while moving the durable vector-sync runtime. Focused source, progress, batch, repository, and end-to-end vector-sync tests cover those paths; no public queue payloads or batch-size defaults changed.

Signed-off-by: phernandez <paul@basicmachines.co>
Signed-off-by: phernandez <paul@basicmachines.co>
@phernandez
phernandez merged commit 3bcb821 into main Jul 21, 2026
22 checks passed
@phernandez
phernandez deleted the codex-1110-vector-planner branch July 21, 2026 05:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant