feat(mcp): leadbay_scan_portfolio_signals — bulk portfolio signal scan + stale_at honesty fix (v0.19.1, product#3704)

.claude-plugin/plugins/leadbay/skills/leadbay_followup_check_in/SKILL.md

@@ -166,6 +166,29 @@ Unlike `leadbay_daily_check_in` which deep-dives on every promising lead in Phas
 
 When the user picks a row, call `leadbay_research_lead_by_id` on that single lead (or `leadbay_research_lead_by_name_fuzzy` if they only have the name) and offer to `leadbay_prepare_outreach` once they say "let's reach out".
 
+# PHASE 3b — BULK SIGNAL SCAN (when the user asks "which of these have signal X")
+
+If the user wants to filter the whole portfolio by a web-research signal — "which of my leads acquired a company since 2025", "find everyone with a funding signal", "who changed CEO" — do NOT loop `leadbay_research_lead_by_id` per row, and do NOT guess from freshness fields. Call `leadbay_scan_portfolio_signals({query, since?})` once: it bulk-reads the cached signals across the portfolio and returns only the matches, campaign-ready. Offer to build a campaign from the matches.
+
+**Close the gap — don't just report it.** The scan returns `not_researched[]` (leads with no cached Leadbay signal) and may surface matches whose cached signal is thin or undated. These are the leads where the answer is genuinely unknown, not absent. Rather than stopping at "K aren't researched yet", offer to fill the gap and refine:
+
+1. **Name the gap precisely** — "N of your M leads matched; K have no cached signal and J more have only a thin/undated mention, so I can't yet confirm signal X for those."
+2. **Run a targeted live pass** — if you have web-search tools, research the specific `not_researched` / thin-signal leads for the exact signal the user asked about (company name + the query terms, e.g. "<Company> acquisition 2025"). Do this only for the gap leads, not the whole portfolio — the cached matches are already answered.
+3. **Fold the findings back in, clearly labelled** — present live results as **agent-sourced (not Leadbay-verified)**, in a section separate from the cached `matched` cohort, and cite the source URL/date you found. Never silently merge a web-found signal into the campaign-ready cohort as if Leadbay had verified it.
+4. **Offer the durable path too** — for gap leads worth persisting, offer `leadbay_bulk_qualify_leads` so Leadbay runs its own `web_fetch` and the signal lands in the portfolio's cached `signals[]` on the next scan.
+
+This keeps the honesty guarantee intact (Leadbay's cached `signals[]` stay the source of truth) while still answering the user's question for the leads Leadbay hasn't researched yet.
+
+**SIGNAL HONESTY — never infer signals from freshness.** `stale_at`,
+`web_fetch_in_progress`, `fetch_at` are freshness markers, not signal
+indicators — signal presence is read ONLY from the actual `signals[]` /
+`web_fetch.content` entries. For "which of my leads have signal X" across a
+portfolio, call **`leadbay_scan_portfolio_signals`** (bulk-reads cached
+signals); don't loop `leadbay_research_lead_by_id` per lead or guess from
+freshness. A lead with no cached content is `not_researched`, not "no match";
+never report a signal verdict for a lead you never read.
+
+
 # CROSS-MODE PIVOT
 
 Below the table, offer the cross-mode pivot in one short line so the user can redirect if you guessed wrong on entry-point routing: "Want to see NEW leads from your wishlist instead?" — that routes back to `leadbay_daily_check_in` (Discovery via `leadbay_pull_leads`).

WORKFLOWS.md

@@ -35,6 +35,7 @@ The table is the human-readable index. The `yaml expected` + `yaml scenario` blo
 | 21 | **Artifact proposal gate** — after a lead batch, agent must offer to build a named artifact | `leadbay_daily_check_in` | "Show me today's leads." |
 | 22 | **Recurrence routing gate** — recurrence language ("I do this every day") must run the daily DISCOVERY check-in, not misroute to follow-ups | `leadbay_daily_check_in` | "Run my morning check-in — I do this every day." |
 | 23 | **Widget overdelivery guard** — when user pre-states full action chain, no "what next?" widget | `leadbay_daily_check_in` | "Show me today's leads and then research the top one for me." |
+| 24 | **Bulk portfolio signal scan** — "which of my leads acquired a company since 2025", "scan my portfolio for funding signals", "find everyone who changed CEO" — filters a known portfolio by a web-research signal in ONE call instead of looping `leadbay_research_lead_by_id` per lead | `leadbay_scan_portfolio_signals` | "Which of my leads acquired a company since 2025?" |
 
 ---
 

packages/core/src/composite/_composite-file-names.ts

@@ -40,6 +40,7 @@ export const COMPOSITE_FILE_TOOL_NAMES: ReadonlySet<string> = new Set([
   "leadbay_research_lead_by_id",
   "leadbay_research_lead_by_name_fuzzy",
   "leadbay_resolve_import_rows",
+  "leadbay_scan_portfolio_signals",
   "leadbay_seed_candidates",
   "leadbay_tour_plan",
 ]);

packages/core/src/composite/_web-fetch-helpers.ts

@@ -0,0 +1,55 @@
+import type { WebFetchSignalsSection } from "../types.js";
+
+// Shared web_fetch.content reshaping. The backend keys web-research signals
+// by emoji-prefixed section labels (e.g. "🏢 company profile", "📈 business
+// signals"); each value is an array of WebFetchEntry. Composites that surface
+// signals to the agent reshape this dynamic dict into an ordered array.
+//
+// Used by leadbay_research_lead_by_id (single lead) and
+// leadbay_scan_portfolio_signals (bulk read).
+
+// Stable section ordering: profile → signals → clues → others (alphabetical).
+export const SECTION_PRIORITY = ["profile", "signals", "clues"];
+
+// Map an emoji-prefixed section label like "🏢 company profile" to
+// {emoji: "🏢", label: "company profile"}. If no emoji prefix, label stays
+// as-is and emoji is null.
+export function splitEmojiSection(key: string): {
+  emoji: string | null;
+  label: string;
+} {
+  // Match a leading non-letter/non-digit run (typically emoji) followed by space.
+  const m = key.match(/^([^\p{L}\p{N}\s]+)\s+(.+)$/u);
+  if (m) return { emoji: m[1], label: m[2] };
+  return { emoji: null, label: key };
+}
+
+export function reshapeWebFetchContent(
+  content: Record<string, unknown> | null
+): WebFetchSignalsSection[] {
+  if (!content) return [];
+  const sections: WebFetchSignalsSection[] = [];
+  for (const [key, val] of Object.entries(content)) {
+    if (!Array.isArray(val)) continue;
+    const { emoji, label } = splitEmojiSection(key);
+    sections.push({
+      section_label: label,
+      section_emoji: emoji,
+      entries: val as WebFetchSignalsSection["entries"],
+    });
+  }
+  // Sort: known section labels first (in priority order), then alphabetical.
+  sections.sort((a, b) => {
+    const ai = SECTION_PRIORITY.findIndex((p) =>
+      a.section_label.toLowerCase().includes(p)
+    );
+    const bi = SECTION_PRIORITY.findIndex((p) =>
+      b.section_label.toLowerCase().includes(p)
+    );
+    const aN = ai < 0 ? SECTION_PRIORITY.length : ai;
+    const bN = bi < 0 ? SECTION_PRIORITY.length : bi;
+    if (aN !== bN) return aN - bN;
+    return a.section_label.localeCompare(b.section_label);
+  });
+  return sections;
+}

packages/core/src/composite/research-lead-by-id.ts

@@ -5,12 +5,12 @@ import type {
   AiAgentResponse,
   LeadPayload,
   LeadWebFetchPayload,
-  WebFetchSignalsSection,
   PaidContactPayload,
   ContactPayload,
   PaginatedActivities,
 } from "../types.js";
 import { withAgentMemoryMeta } from "../agent-memory/index.js";
+import { reshapeWebFetchContent } from "./_web-fetch-helpers.js";
 
 import { leadbay_research_lead_by_id as RESEARCH_LEAD_BY_ID_DESCRIPTION } from "../tool-descriptions.generated.js";
 
@@ -169,43 +169,8 @@ export function renderResearchLeadMarkdown(
   return out.join("\n");
 }
 
-// Map an emoji-prefixed section label like "🏢 company profile" to
-// {section_emoji: "🏢", section_label: "company profile"}. If no emoji, label
-// stays as-is. Stable section ordering: profile → signals → clues → others.
-const SECTION_PRIORITY = ["profile", "signals", "clues"];
-
-function splitEmojiSection(key: string): { emoji: string | null; label: string } {
-  // Match a leading non-letter/non-digit character (typically emoji) followed by space.
-  const m = key.match(/^([^\p{L}\p{N}\s]+)\s+(.+)$/u);
-  if (m) return { emoji: m[1], label: m[2] };
-  return { emoji: null, label: key };
-}
-
-function reshapeWebFetchContent(
-  content: Record<string, unknown> | null
-): WebFetchSignalsSection[] {
-  if (!content) return [];
-  const sections: WebFetchSignalsSection[] = [];
-  for (const [key, val] of Object.entries(content)) {
-    if (!Array.isArray(val)) continue;
-    const { emoji, label } = splitEmojiSection(key);
-    sections.push({
-      section_label: label,
-      section_emoji: emoji,
-      entries: val as WebFetchSignalsSection["entries"],
-    });
-  }
-  // Sort: known section labels first (in priority order), then alphabetical.
-  sections.sort((a, b) => {
-    const ai = SECTION_PRIORITY.findIndex((p) => a.section_label.toLowerCase().includes(p));
-    const bi = SECTION_PRIORITY.findIndex((p) => b.section_label.toLowerCase().includes(p));
-    const aN = ai < 0 ? SECTION_PRIORITY.length : ai;
-    const bN = bi < 0 ? SECTION_PRIORITY.length : bi;
-    if (aN !== bN) return aN - bN;
-    return a.section_label.localeCompare(b.section_label);
-  });
-  return sections;
-}
+// Section reshaping (splitEmojiSection / reshapeWebFetchContent / SECTION_PRIORITY)
+// moved to ./_web-fetch-helpers.js — shared with leadbay_scan_portfolio_signals.
 
 // Hashable contact summary used by hasReachableContact. A contact is
 // "reachable" iff it has at least one of: non-empty email, non-empty