fix(fuzzy): search full wishlist, not just top 50#99
Draft
ArtyETH06 wants to merge 1 commit into
Draft
Conversation
leadbay_research_lead_by_name_fuzzy fetched only page 0 (top 50) of the
active lens wishlist, so any lead beyond the first page returned a
spurious LEAD_NOT_FOUND. MCP telemetry (7d ending 2026-06-11) showed
17/17 calls failing with LEAD_NOT_FOUND across three consecutive
analysis runs.
Root cause: research-lead-by-name-fuzzy.ts only called discoverLeads with
{count:50, page:0}. The wishlist is paginated (and NOT strictly
score-ordered — a high-score lead can land on page 1), so the substring
matcher never saw leads beyond the top-50 window.
Fix: page through the full wishlist (capped at 40 pages / 2000 leads)
before ranking. Updated the inputSchema description, the LEAD_NOT_FOUND
message, and the promptforge template accordingly.
Reproduce-as-eval: WORKFLOWS.md workflow #25 ("Research the company
YOUR CAR TODAY LLC") — a real lead that lives on page 1 of the active
test lens (score 86, total=51). Verified live: pre-fix returns
LEAD_NOT_FOUND, post-fix resolves with full firmographics. New unit test
research-lead-by-name-fuzzy-pagination.test.ts pins the page-1 case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Author
|
TEST PR from the mcp-analysis / eval skill |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
leadbay_research_lead_by_name_fuzzyfailed 100% of the time. MCP telemetry (7d ending 2026-06-11): 17 calls, 0 ok, 17 LEAD_NOT_FOUND, recurring across three consecutive MCP-analysis runs (2026-06-02, 2026-06-09, 2026-06-11). A perfect-zero rate is a real defect, not a scope limit.Root cause
packages/core/src/composite/research-lead-by-name-fuzzy.tsfetched only page 0 (top 50) of the active lens wishlist (discoverLeads.execute(client, {count:50, page:0})), then substring-matched against that window. Any lead beyond the first page was invisible to the matcher.Crucially, the wishlist is not strictly score-ordered — a high-score lead can land on page 1. Live probe of the test account confirmed: the active lens (
last_requested_lens= 40005) hastotal=51, and "YOUR CAR TODAY LLC" (firmographic score 86) sits on page 1, absent from page 0. The default lens (39107) hastotal=60with 10 leads stranded on page 1. Every lookup of such a lead returned a spuriousLEAD_NOT_FOUND.Fix
Page through the full wishlist (capped at 40 pages / 2000 leads) before ranking, breaking when
has_moreis false. Updated thecompanyNameinputSchema description, theLEAD_NOT_FOUNDhint ("searched the full wishlist"), and the promptforge template; regeneratedtool-descriptions.generated.ts.Reproduce-as-eval
Added WORKFLOWS.md workflow #25 — "Fuzzy lead lookup beyond the top page" (scenario: "Research the company YOUR CAR TODAY LLC for me."), a real lead on page 1 of the active test lens. Plus a new unit test
research-lead-by-name-fuzzy-pagination.test.tspinning the page-1 case (mocked).Verification
pnpm -r typecheck— green (4/4 packages)pnpm -r test— green (core 381, promptforge 16, mcp 382)/eval --workflow 25— PASS (5/5/5/5), 3 rounds. Rounds 1-2 failed for scenario-quality reasons only (round 1 used a company from the wrong lens; round 2's "YOUR CAR TODAY" parsed as "do you have a car?"). The fix itself was correct throughout — round 3's unambiguous phrasing confirmed the fuzzy tool resolves the page-1 lead with full firmographics (_meta.resolved_from="companyName", score 86) instead ofLEAD_NOT_FOUND. The judge scored mission_match / instruction_adherence / no_fabrication / tool_selection_fit all 5/5; invariants 2/2 PASS.Files changed
packages/core/src/composite/research-lead-by-name-fuzzy.ts— full-wishlist pagination (the fix)packages/promptforge/tool-descriptions/composite/research-lead-by-name-fuzzy.md.tmpl— description updatepackages/core/src/tool-descriptions.generated.ts— regeneratedpackages/core/test/unit/composite/research-lead-by-name-fuzzy-pagination.test.ts— new regression testWORKFLOWS.md— workflow Publish Leadbay MCP to marketplaces #25 contract + scenario