feat(mcp): background incremental registry sync

[bistack]

Summary

registry_sync no longer blocks on (or even performs) a full registry download on every call:

• Cache-first fast return: a fresh local snapshot is served with zero network requests; downloads run fully in the background via tokio::spawn, guarded by a process-wide mutex so at most one sync runs at a time.
• Incremental refresh: after the first full snapshot, refresh uses the upstream updated_since filter (1-3 pages vs 46+ for the full listing), with a monthly full pagination as reconciliation when the snapshot is missing or older than FULL_RESYNC_INTERVAL_SECS.
• Atomic replace: the next snapshot is assembled in memory and written via temp-file + rename (existing write_atomic), so a failed sync never corrupts or loses the previous snapshot.
• Pacing + User-Agent: 500ms inter-page delay and a proper UA for the flaky preview API (measured 20s+ stalls on some pages).

Source-structure budget

scripts/source-structure-budget.json bumps max_total_owned_rust_lines 673375 -> 673820 (+445 net lines; +507/-62 in mcp_registry.rs). Composition: ~160 lines of production code (background sync via tokio::spawn, updated_since incremental refresh with in-memory merge, atomic cache replace, the cache-path injection hook), ~305 lines of tests (sync_strategy / merge_incremental_entries / catalog coverage, fixture updates, the cross-platform cache-path test hook), ~45 lines of comments. No new modules; growth is in the existing mcp_registry.rs module.

Testing

• cargo fmt --all -- --check
• cargo clippy --workspace --all-features --locked (warning-free under the CI allow list)
• cargo test -p codewhale-tui --locked mcp_registry — all 21 tests pass on macOS/Linux, including the previously Windows-failing fresh_cache_serves_catalog_without_network (fixed via an explicit cache-path injection, because dirs::home_dir() resolves the OS profile directory on Windows through SHGetKnownFolderPath, which no environment variable can redirect).

CI status (as of the latest run)

• Windows: the only failing test is headless_bash_success_and_failure_are_distinct_bounded_exact_evidence (crates/tui/tests/adaptive_evidence_acceptance.rs), an unrelated pre-existing Windows issue: write_session_artifact fails on Windows, the truncation footer falls back to the legacy path, and the assertion receipt.contains("/artifacts/") fails. This test was previously masked by --fail-fast (the old fresh_cache failure stopped the run before integration tests) and is not touched by this PR. All mcp_registry tests pass on Windows.
• macOS: cargo test itself passes; the job stalls on the "Run isolated Skills Manager PTY acceptance" step, a pre-existing PTY acceptance hang documented in CONTRIBUTING (input-starvation issue), unrelated to this PR.
• Ubuntu: pass.

Checklist

• Updated docs or comments as needed
• Added or updated tests where relevant (21 unit tests in mcp_registry)
• Verified TUI behavior manually if UI changes — no UI changes (backend tool only)
• Harvested/co-authored credit uses a GitHub numeric noreply address — N/A (no co-authors)

No-Issue: MCP registry sync improvement; no tracked issue yet

[github-actions]

Thanks @bistack for taking the time to contribute.

This repository is observing a maintainer-managed PR intake gate in dry-run mode, so this pull request is staying open. This note helps maintainers prepare the allowlist before any enforcement is considered.

Please read CONTRIBUTING.md for the expected contribution shape. A maintainer can grant recurring PR access by commenting /lgtm on a pull request.