Skip to content

Commit a2a6d7d

Browse files
prasjaiswalPrashant Jaiswalclaude
authored
Use LLM-based scoring for platform handler queries (#71)
Platform-retrieved items (Shopify/Wix MCP) were all scoring below the min_score=51 threshold with the default Pi Labs scorer, resulting in empty results. Override to 4.1-mini scorer for KeywordRetrievalHandler queries using the existing override_scoring_provider context manager (same pattern as aajtak.py). Pi Labs remains the default scorer for vector DB queries. Co-authored-by: Prashant Jaiswal <prashjai@Prashants-MacBook-Pro.local> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 3936d55 commit a2a6d7d

File tree

1 file changed

+14
-10
lines changed
  • ask_api/packages/providers/platform_handlers/nlweb_platform_handlers

1 file changed

+14
-10
lines changed

ask_api/packages/providers/platform_handlers/nlweb_platform_handlers/base.py

Lines changed: 14 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@
1414
import logging
1515
from abc import abstractmethod
1616

17+
from nlweb_core.config import override_scoring_provider
1718
from nlweb_core.handler import DefaultAskHandler, OutputMethod
1819
from nlweb_core.protocol.models import AskRequest
1920
from nlweb_core.retrieved_item import RetrievedItem
@@ -64,16 +65,19 @@ async def _run_query_body(
6465
num_results=request.query.num_results,
6566
)
6667

67-
# Use the same ranking pipeline as DefaultAskHandler
68-
final_ranked_answers = await Ranking().rank(
69-
items=retrieved_items,
70-
query_text=query,
71-
item_type=site_config["item_type"],
72-
max_results=request.query.max_results,
73-
min_score=request.query.min_score,
74-
site=request.query.site,
75-
start_num=self._get_result_offset(request),
76-
)
68+
# Use LLM-based scoring for platform-retrieved items.
69+
# Platform search APIs return pre-filtered results that need LLM evaluation
70+
# rather than the default Pi Labs scorer which under-scores these items.
71+
with override_scoring_provider("default", "4.1-mini"):
72+
final_ranked_answers = await Ranking().rank(
73+
items=retrieved_items,
74+
query_text=query,
75+
item_type=site_config["item_type"],
76+
max_results=request.query.max_results,
77+
min_score=request.query.min_score,
78+
site=request.query.site,
79+
start_num=self._get_result_offset(request),
80+
)
7781

7882
await self._send_results(output_method, final_ranked_answers)
7983
return final_ranked_answers

0 commit comments

Comments
 (0)