diff --git a/.claude-plugin/plugins/leadbay/skills/leadbay_followup_check_in/SKILL.md b/.claude-plugin/plugins/leadbay/skills/leadbay_followup_check_in/SKILL.md index 56a21ca8..13618ba4 100644 --- a/.claude-plugin/plugins/leadbay/skills/leadbay_followup_check_in/SKILL.md +++ b/.claude-plugin/plugins/leadbay/skills/leadbay_followup_check_in/SKILL.md @@ -166,6 +166,29 @@ Unlike `leadbay_daily_check_in` which deep-dives on every promising lead in Phas When the user picks a row, call `leadbay_research_lead_by_id` on that single lead (or `leadbay_research_lead_by_name_fuzzy` if they only have the name) and offer to `leadbay_prepare_outreach` once they say "let's reach out". +# PHASE 3b — BULK SIGNAL SCAN (when the user asks "which of these have signal X") + +If the user wants to filter the whole portfolio by a web-research signal — "which of my leads acquired a company since 2025", "find everyone with a funding signal", "who changed CEO" — do NOT loop `leadbay_research_lead_by_id` per row, and do NOT guess from freshness fields. Call `leadbay_scan_portfolio_signals({query, since?})` once: it bulk-reads the cached signals across the portfolio and returns only the matches, campaign-ready. Offer to build a campaign from the matches. + +**Close the gap — don't just report it.** The scan returns `not_researched[]` (leads with no cached Leadbay signal) and may surface matches whose cached signal is thin or undated. These are the leads where the answer is genuinely unknown, not absent. Rather than stopping at "K aren't researched yet", offer to fill the gap and refine: + +1. **Name the gap precisely** — "N of your M leads matched; K have no cached signal and J more have only a thin/undated mention, so I can't yet confirm signal X for those." +2. **Run a targeted live pass** — if you have web-search tools, research the specific `not_researched` / thin-signal leads for the exact signal the user asked about (company name + the query terms, e.g. " acquisition 2025"). Do this only for the gap leads, not the whole portfolio — the cached matches are already answered. +3. **Fold the findings back in, clearly labelled** — present live results as **agent-sourced (not Leadbay-verified)**, in a section separate from the cached `matched` cohort, and cite the source URL/date you found. Never silently merge a web-found signal into the campaign-ready cohort as if Leadbay had verified it. +4. **Offer the durable path too** — for gap leads worth persisting, offer `leadbay_bulk_qualify_leads` so Leadbay runs its own `web_fetch` and the signal lands in the portfolio's cached `signals[]` on the next scan. + +This keeps the honesty guarantee intact (Leadbay's cached `signals[]` stay the source of truth) while still answering the user's question for the leads Leadbay hasn't researched yet. + +**SIGNAL HONESTY — never infer signals from freshness.** `stale_at`, +`web_fetch_in_progress`, `fetch_at` are freshness markers, not signal +indicators — signal presence is read ONLY from the actual `signals[]` / +`web_fetch.content` entries. For "which of my leads have signal X" across a +portfolio, call **`leadbay_scan_portfolio_signals`** (bulk-reads cached +signals); don't loop `leadbay_research_lead_by_id` per lead or guess from +freshness. A lead with no cached content is `not_researched`, not "no match"; +never report a signal verdict for a lead you never read. + + # CROSS-MODE PIVOT Below the table, offer the cross-mode pivot in one short line so the user can redirect if you guessed wrong on entry-point routing: "Want to see NEW leads from your wishlist instead?" — that routes back to `leadbay_daily_check_in` (Discovery via `leadbay_pull_leads`). diff --git a/WORKFLOWS.md b/WORKFLOWS.md index 4efb6863..41a2ce7b 100644 --- a/WORKFLOWS.md +++ b/WORKFLOWS.md @@ -35,6 +35,7 @@ The table is the human-readable index. The `yaml expected` + `yaml scenario` blo | 21 | **Artifact proposal gate** — after a lead batch, agent must offer to build a named artifact | `leadbay_daily_check_in` | "Show me today's leads." | | 22 | **Recurrence routing gate** — recurrence language ("I do this every day") must run the daily DISCOVERY check-in, not misroute to follow-ups | `leadbay_daily_check_in` | "Run my morning check-in — I do this every day." | | 23 | **Widget overdelivery guard** — when user pre-states full action chain, no "what next?" widget | `leadbay_daily_check_in` | "Show me today's leads and then research the top one for me." | +| 24 | **Bulk portfolio signal scan** — "which of my leads acquired a company since 2025", "scan my portfolio for funding signals", "find everyone who changed CEO" — filters a known portfolio by a web-research signal in ONE call instead of looping `leadbay_research_lead_by_id` per lead | `leadbay_scan_portfolio_signals` | "Which of my leads acquired a company since 2025?" | --- diff --git a/packages/core/src/composite/_composite-file-names.ts b/packages/core/src/composite/_composite-file-names.ts index a6c57ce6..6355e7bf 100644 --- a/packages/core/src/composite/_composite-file-names.ts +++ b/packages/core/src/composite/_composite-file-names.ts @@ -40,6 +40,7 @@ export const COMPOSITE_FILE_TOOL_NAMES: ReadonlySet = new Set([ "leadbay_research_lead_by_id", "leadbay_research_lead_by_name_fuzzy", "leadbay_resolve_import_rows", + "leadbay_scan_portfolio_signals", "leadbay_seed_candidates", "leadbay_tour_plan", ]); diff --git a/packages/core/src/composite/_web-fetch-helpers.ts b/packages/core/src/composite/_web-fetch-helpers.ts new file mode 100644 index 00000000..1c7a8375 --- /dev/null +++ b/packages/core/src/composite/_web-fetch-helpers.ts @@ -0,0 +1,55 @@ +import type { WebFetchSignalsSection } from "../types.js"; + +// Shared web_fetch.content reshaping. The backend keys web-research signals +// by emoji-prefixed section labels (e.g. "🏢 company profile", "📈 business +// signals"); each value is an array of WebFetchEntry. Composites that surface +// signals to the agent reshape this dynamic dict into an ordered array. +// +// Used by leadbay_research_lead_by_id (single lead) and +// leadbay_scan_portfolio_signals (bulk read). + +// Stable section ordering: profile → signals → clues → others (alphabetical). +export const SECTION_PRIORITY = ["profile", "signals", "clues"]; + +// Map an emoji-prefixed section label like "🏢 company profile" to +// {emoji: "🏢", label: "company profile"}. If no emoji prefix, label stays +// as-is and emoji is null. +export function splitEmojiSection(key: string): { + emoji: string | null; + label: string; +} { + // Match a leading non-letter/non-digit run (typically emoji) followed by space. + const m = key.match(/^([^\p{L}\p{N}\s]+)\s+(.+)$/u); + if (m) return { emoji: m[1], label: m[2] }; + return { emoji: null, label: key }; +} + +export function reshapeWebFetchContent( + content: Record | null +): WebFetchSignalsSection[] { + if (!content) return []; + const sections: WebFetchSignalsSection[] = []; + for (const [key, val] of Object.entries(content)) { + if (!Array.isArray(val)) continue; + const { emoji, label } = splitEmojiSection(key); + sections.push({ + section_label: label, + section_emoji: emoji, + entries: val as WebFetchSignalsSection["entries"], + }); + } + // Sort: known section labels first (in priority order), then alphabetical. + sections.sort((a, b) => { + const ai = SECTION_PRIORITY.findIndex((p) => + a.section_label.toLowerCase().includes(p) + ); + const bi = SECTION_PRIORITY.findIndex((p) => + b.section_label.toLowerCase().includes(p) + ); + const aN = ai < 0 ? SECTION_PRIORITY.length : ai; + const bN = bi < 0 ? SECTION_PRIORITY.length : bi; + if (aN !== bN) return aN - bN; + return a.section_label.localeCompare(b.section_label); + }); + return sections; +} diff --git a/packages/core/src/composite/research-lead-by-id.ts b/packages/core/src/composite/research-lead-by-id.ts index 76e94aca..349187ce 100644 --- a/packages/core/src/composite/research-lead-by-id.ts +++ b/packages/core/src/composite/research-lead-by-id.ts @@ -5,12 +5,12 @@ import type { AiAgentResponse, LeadPayload, LeadWebFetchPayload, - WebFetchSignalsSection, PaidContactPayload, ContactPayload, PaginatedActivities, } from "../types.js"; import { withAgentMemoryMeta } from "../agent-memory/index.js"; +import { reshapeWebFetchContent } from "./_web-fetch-helpers.js"; import { leadbay_research_lead_by_id as RESEARCH_LEAD_BY_ID_DESCRIPTION } from "../tool-descriptions.generated.js"; @@ -169,43 +169,8 @@ export function renderResearchLeadMarkdown( return out.join("\n"); } -// Map an emoji-prefixed section label like "🏢 company profile" to -// {section_emoji: "🏢", section_label: "company profile"}. If no emoji, label -// stays as-is. Stable section ordering: profile → signals → clues → others. -const SECTION_PRIORITY = ["profile", "signals", "clues"]; - -function splitEmojiSection(key: string): { emoji: string | null; label: string } { - // Match a leading non-letter/non-digit character (typically emoji) followed by space. - const m = key.match(/^([^\p{L}\p{N}\s]+)\s+(.+)$/u); - if (m) return { emoji: m[1], label: m[2] }; - return { emoji: null, label: key }; -} - -function reshapeWebFetchContent( - content: Record | null -): WebFetchSignalsSection[] { - if (!content) return []; - const sections: WebFetchSignalsSection[] = []; - for (const [key, val] of Object.entries(content)) { - if (!Array.isArray(val)) continue; - const { emoji, label } = splitEmojiSection(key); - sections.push({ - section_label: label, - section_emoji: emoji, - entries: val as WebFetchSignalsSection["entries"], - }); - } - // Sort: known section labels first (in priority order), then alphabetical. - sections.sort((a, b) => { - const ai = SECTION_PRIORITY.findIndex((p) => a.section_label.toLowerCase().includes(p)); - const bi = SECTION_PRIORITY.findIndex((p) => b.section_label.toLowerCase().includes(p)); - const aN = ai < 0 ? SECTION_PRIORITY.length : ai; - const bN = bi < 0 ? SECTION_PRIORITY.length : bi; - if (aN !== bN) return aN - bN; - return a.section_label.localeCompare(b.section_label); - }); - return sections; -} +// Section reshaping (splitEmojiSection / reshapeWebFetchContent / SECTION_PRIORITY) +// moved to ./_web-fetch-helpers.js — shared with leadbay_scan_portfolio_signals. // Hashable contact summary used by hasReachableContact. A contact is // "reachable" iff it has at least one of: non-empty email, non-empty diff --git a/packages/core/src/composite/scan-portfolio-signals.ts b/packages/core/src/composite/scan-portfolio-signals.ts new file mode 100644 index 00000000..86ee96e2 --- /dev/null +++ b/packages/core/src/composite/scan-portfolio-signals.ts @@ -0,0 +1,461 @@ +import type { LeadbayClient } from "../client.js"; +import type { + Tool, + ToolContext, + MonitorFilterItem, + LeadWebFetchPayload, + WebFetchEntry, +} from "../types.js"; +import { withAgentMemoryMeta } from "../agent-memory/index.js"; +import { reshapeWebFetchContent } from "./_web-fetch-helpers.js"; +import { resolveLocations } from "./_geo-helpers.js"; + +import { leadbay_scan_portfolio_signals as SCAN_PORTFOLIO_SIGNALS_DESCRIPTION } from "../tool-descriptions.generated.js"; + +// Bulk portfolio signal scan. Reads CACHED web-research signals across a +// Monitor portfolio (or an explicit lead-id list) and returns only the leads +// whose signals match a free-text query — e.g. "M&A", "acquisition", +// "racheté". This is the read-only, no-quota counterpart to looping +// leadbay_research_lead_by_id one lead at a time. +// +// Hard invariant (issue #3704): the tool searches ONLY actual web_fetch +// content. It never infers signal presence/absence from freshness markers +// (stale_at / web_fetch_in_progress / fetch_at). Leads with no cached content +// land in `not_researched[]` — they are NOT silently treated as "no match". + +const DEFAULT_MAX_LEADS = 200; +const HARD_MAX_LEADS = 300; +const MONITOR_PAGE_SIZE = 200; // backend cap for /monitor count + +interface ScanPortfolioSignalsParams { + query: string; + leadIds?: string[]; + city?: string; + city_id?: string; + set_filter?: MonitorFilterItem; + since?: string; + max_leads?: number; +} + +interface MatchedSignal { + section_label: string; + section_emoji: string | null; + hot: boolean; + source: string; + date: string | null; + description: string; +} + +interface MatchedLead { + lead_id: string; + name: string | null; + location: string | null; + matched_signals: MatchedSignal[]; +} + +interface MonitorResponse { + items?: any[]; + leads?: any[]; + pagination?: any; + [k: string]: unknown; +} + +// Diacritic-fold + lowercase so "racheté" matches query "rachat" and "M&A" +// matches "m&a". NFD splits accented chars into base + combining mark; we +// strip the marks (U+0300–U+036F). +function fold(s: string): string { + return s + .normalize("NFD") + .replace(/[̀-ͯ]/g, "") + .toLowerCase() + .trim(); +} + +// Split the query into OR terms on commas / whitespace, folded. Empty terms +// dropped. An all-whitespace query yields [] → matches nothing. +function parseQueryTerms(query: string): string[] { + return query + .split(/[,\s]+/) + .map((t) => fold(t)) + .filter((t) => t.length > 0); +} + +// Monitor leads carry `location` as either a plain string or an object +// ({city, state, country, full, pos}). Render a compact "City, State" (or the +// `full` string, or the bare string) for the place-card blocks — never the +// raw object or the `pos` coordinates. +function shortLocation(loc: unknown): string | null { + if (loc == null) return null; + if (typeof loc === "string") return loc.trim() || null; + if (typeof loc === "object") { + const o = loc as Record; + const clean = (v: unknown) => { + const s = typeof v === "string" ? v.trim() : ""; + return s && s.toUpperCase() !== "N/A" ? s : ""; + }; + const city = clean(o.city); + const state = clean(o.state); + if (city && state) return `${city}, ${state}`; + if (city) return city; + if (typeof o.full === "string" && o.full.trim()) return o.full.trim(); + } + return null; +} + +function mergeLocationIds( + filter: MonitorFilterItem | undefined, + ids: string[] +): MonitorFilterItem { + const criteria: Array> = filter?.criteria + ? [...filter.criteria] + : []; + const idx = criteria.findIndex( + (c) => c?.type === "location_ids" && c?.is_excluded === false + ); + if (idx >= 0) { + const cur = criteria[idx]; + const existing = Array.isArray(cur.locations) + ? (cur.locations as string[]) + : []; + const merged = Array.from(new Set([...existing, ...ids])); + criteria[idx] = { ...cur, locations: merged }; + } else { + criteria.push({ type: "location_ids", is_excluded: false, locations: ids }); + } + return { criteria }; +} + +// Does this signal entry match any query term? Searches description, source, +// and the section label so "funding" matches a "📈 funding" section header too. +function entryMatches( + entry: WebFetchEntry, + sectionLabel: string, + terms: string[] +): boolean { + if (terms.length === 0) return false; + const haystack = fold( + [entry.description ?? "", entry.source ?? "", sectionLabel].join("  ") + ); + return terms.some((t) => haystack.includes(t)); +} + +// Keep only entries on/after `since` (ISO date). Entries with no parseable +// date are KEPT — absence of a date is not evidence the event is old, and +// dropping them would silently hide real matches (issue #3704 honesty rule). +function passesSince(entry: WebFetchEntry, sinceMs: number | null): boolean { + if (sinceMs == null) return true; + if (!entry.date) return true; + const ts = Date.parse(entry.date); + if (Number.isNaN(ts)) return true; + return ts >= sinceMs; +} + +export const scanPortfolioSignals: Tool = { + name: "leadbay_scan_portfolio_signals", + annotations: { + title: "Scan a portfolio for a web-research signal in bulk", + readOnlyHint: true, + destructiveHint: false, + idempotentHint: true, + openWorldHint: true, + }, + description: SCAN_PORTFOLIO_SIGNALS_DESCRIPTION, + inputSchema: { + type: "object", + properties: { + query: { + type: "string", + description: + "Signal terms to match (case- and accent-insensitive). Comma- or space-separated terms are OR'd, e.g. 'M&A, acquisition, racheté'. Matched against each signal entry's description, source, and section label.", + }, + leadIds: { + type: "array", + items: { type: "string" }, + description: + "Explicit lead UUIDs to scan (skips Monitor pagination). Use when you already hold a cohort of ids.", + }, + city: { + type: "string", + description: + "Free-text city / region to scope the Monitor portfolio before scanning (resolved via /geo/search, same as leadbay_pull_followups). Ignored when `leadIds` is given.", + }, + city_id: { + type: "string", + description: + "Pre-resolved admin_area id (numeric string). Bypasses the resolver. Ignored when `leadIds` is given.", + }, + set_filter: { + type: "object", + description: + "Optional Monitor FilterItem ({criteria: FilterCriterion[]}) to scope the portfolio before scanning. Persisted server-side then applied, mirroring leadbay_pull_followups. Ignored when `leadIds` is given.", + properties: { + criteria: { type: "array", items: { type: "object" } }, + }, + }, + since: { + type: "string", + description: + "ISO date (e.g. '2025-01-01'). When set, only signal entries dated on/after it are returned. Entries with no date are kept (absence of a date is not evidence the event is old).", + }, + max_leads: { + type: "number", + description: `Cap on leads scanned (default ${DEFAULT_MAX_LEADS}, hard max ${HARD_MAX_LEADS}). When the portfolio exceeds this, the scan is truncated and truncated_at is set.`, + }, + }, + required: ["query"], + additionalProperties: false, + }, + outputSchema: { + type: "object", + properties: { + matched: { + type: "array", + description: + "Leads with ≥1 signal entry matching the query. Each: {lead_id, name, location, matched_signals:[{section_label, section_emoji, hot, source, date, description}]}. Campaign-ready — feed lead_ids straight into leadbay_add_leads_to_campaign.", + items: { type: "object" }, + }, + not_researched: { + type: "array", + description: + "Leads scanned that had NO cached signal content (web_fetch.content null or still in progress). These are NOT 'no match' — they were never researched. Qualify them (leadbay_bulk_qualify_leads) then re-scan. Each: {lead_id, name}.", + items: { type: "object" }, + }, + scanned_count: { + type: "number", + description: "Total leads read in this scan (matched + non-matching + not_researched).", + }, + matched_count: { type: "number", description: "Length of `matched`." }, + truncated_at: { + type: "number", + description: + "Present only when the portfolio exceeded `max_leads`; equals the cap applied. Coverage is partial — narrow the scope (city / set_filter) or raise max_leads.", + }, + quota_exceeded: { + type: "boolean", + description: + "True if a 429 was hit mid-scan. Partial `matched` is still returned. Offer wait-for-reset OR top-up.", + }, + status: { + type: "string", + description: + "`ambiguous_locations` when a passed `city` matched multiple admin_areas; pick an id from `location_ambiguities` and re-call with `city_id`. Absent on the happy path.", + }, + location_ambiguities: { + type: "array", + description: "Only present when status === 'ambiguous_locations'.", + items: { type: "object" }, + }, + _meta: { + type: "object", + properties: { + region: { type: "string" }, + agent_memory: { type: "object" }, + }, + }, + }, + required: ["matched", "not_researched", "scanned_count", "matched_count", "quota_exceeded"], + }, + execute: async ( + client: LeadbayClient, + params: ScanPortfolioSignalsParams, + ctx?: ToolContext + ) => { + const terms = parseQueryTerms(params.query ?? ""); + const maxLeads = Math.min( + params.max_leads ?? DEFAULT_MAX_LEADS, + HARD_MAX_LEADS + ); + const sinceParsed = params.since ? Date.parse(params.since) : NaN; + const sinceValid = Number.isNaN(sinceParsed) ? null : sinceParsed; + + // ── 1. Resolve scope → ordered list of {id, name, location} ──────────── + let portfolio: Array<{ id: string; name: string | null; location: string | null }>; + let truncatedAt: number | undefined; + // Set true by EITHER a 429 while paging /monitor (can't enumerate the + // portfolio) OR a 429 while reading a lead's web_fetch. Honest signal that + // coverage is partial because of a quota wall, never reported as "no + // matches" (issue #3704). + let quotaExceeded = false; + + if (params.leadIds && params.leadIds.length > 0) { + const sliced = params.leadIds.slice(0, maxLeads); + if (params.leadIds.length > maxLeads) truncatedAt = maxLeads; + portfolio = sliced.map((id) => ({ id, name: null, location: null })); + } else { + // Geo / filter scope, then paginate /monitor (same store-then-apply + // mechanism as leadbay_pull_followups). + let effectiveSetFilter: MonitorFilterItem | undefined = params.set_filter; + const geoTexts: string[] = []; + if (params.city) geoTexts.push(params.city); + if (params.city_id) geoTexts.push(params.city_id); + if (geoTexts.length > 0) { + const { resolved, ambiguities } = await resolveLocations(client, geoTexts); + if (ambiguities.length > 0) { + return withAgentMemoryMeta( + client, + { + status: "ambiguous_locations" as const, + location_ambiguities: ambiguities, + matched: [], + not_researched: [], + scanned_count: 0, + matched_count: 0, + quota_exceeded: false, + _meta: { region: client.region }, + }, + ctx + ); + } + if (resolved.length > 0) { + effectiveSetFilter = mergeLocationIds(effectiveSetFilter, resolved); + } + } + + // Only request `filtered=true` if we actually stored the filter this + // call. If the POST fails, sending `filtered=true` would scan against + // whatever filter was previously persisted server-side — a stale, + // convincing-but-wrong cohort with no visible error. On failure we fall + // back to an UNfiltered scan (honest: wider, not silently-wrong) and + // surface a 429 via quota_exceeded. + let filterStored = false; + if (effectiveSetFilter) { + try { + await client.requestVoid("POST", "/monitor/filter", effectiveSetFilter); + filterStored = true; + } catch (err: any) { + if (err?.code === "QUOTA_EXCEEDED") quotaExceeded = true; + ctx?.logger?.warn?.( + `scan_portfolio_signals: POST /monitor/filter failed (${err?.code ?? err?.message ?? err}); scanning UNfiltered to avoid trusting a stale server-side filter` + ); + } + } + + portfolio = []; + let page = 0; + while (portfolio.length < maxLeads) { + const qs = new URLSearchParams({ + personal: "false", + liked: "false", + filtered: String(filterStored), + count: String(MONITOR_PAGE_SIZE), + page: String(page), + }).toString(); + let monitor: MonitorResponse; + try { + monitor = await client.request("GET", `/monitor?${qs}`); + } catch (err: any) { + if (err?.code === "QUOTA_EXCEEDED") { + // Couldn't finish paging the portfolio — surface what we have and + // flag the quota wall so the agent reports "scan incomplete", not + // "no matches". + quotaExceeded = true; + break; + } + throw err; + } + const rawLeads: any[] = Array.isArray(monitor.items) + ? monitor.items + : Array.isArray(monitor.leads) + ? monitor.leads + : Array.isArray(monitor) + ? (monitor as unknown as any[]) + : []; + if (rawLeads.length === 0) break; + for (const lead of rawLeads) { + if (portfolio.length >= maxLeads) break; + portfolio.push({ + id: lead.id, + name: lead.name ?? null, + location: shortLocation(lead.location), + }); + } + const pages = monitor.pagination?.pages; + if (typeof pages === "number" && page >= pages - 1) break; + if (rawLeads.length < MONITOR_PAGE_SIZE) break; + page += 1; + } + // If we filled the cap, the portfolio may have more leads behind it — + // surface partial coverage. (Monitor's reported total isn't reliable + // across backend versions, so we key off the cap hit.) + if (portfolio.length >= maxLeads) truncatedAt = maxLeads; + } + + // ── 2. Read-only fan-out: GET /leads/{id}/web_fetch (NO POST) ────────── + // The client semaphore caps concurrency at 5; Promise.all is fine. We + // catch per-lead (not Promise.allSettled-then-inspect) so a rejected read + // still carries its `lead` — a failed read must land in not_researched + // with the lead name intact, never silently vanish (issue #3704 honesty + // invariant: scanned_count = matched + non-matching + not_researched). + const matched: MatchedLead[] = []; + const notResearched: Array<{ lead_id: string; name: string | null }> = []; + + const reads = await Promise.all( + portfolio.map(async (lead) => { + try { + const wf = await client.request( + "GET", + `/leads/${lead.id}/web_fetch` + ); + return { lead, wf, error: null as any }; + } catch (error: any) { + // Any read failure (404, 429, network) → we couldn't read signals + // for this lead. Carry the lead through so it lands in + // not_researched: honest "no data read", never a silent "no match". + return { lead, wf: null, error }; + } + }) + ); + + for (const r of reads) { + const { lead, wf, error } = r; + if (error) { + if (error?.code === "QUOTA_EXCEEDED") quotaExceeded = true; + notResearched.push({ lead_id: lead.id, name: lead.name }); + continue; + } + const hasContent = + wf && wf.content != null && wf.in_progress !== true && Object.keys(wf.content).length > 0; + if (!hasContent) { + notResearched.push({ lead_id: lead.id, name: lead.name }); + continue; + } + // Reshape + filter entries by query + since. + const sections = reshapeWebFetchContent(wf.content as Record); + const matchedSignals: MatchedSignal[] = []; + for (const sec of sections) { + for (const entry of sec.entries) { + if (!entryMatches(entry, sec.section_label, terms)) continue; + if (!passesSince(entry, sinceValid)) continue; + matchedSignals.push({ + section_label: sec.section_label, + section_emoji: sec.section_emoji, + hot: entry.hot === true, + source: entry.source ?? "", + date: entry.date ?? null, + description: entry.description ?? "", + }); + } + } + if (matchedSignals.length > 0) { + matched.push({ + lead_id: lead.id, + name: lead.name, + location: lead.location, + matched_signals: matchedSignals, + }); + } + } + + const out: Record = { + matched, + not_researched: notResearched, + scanned_count: portfolio.length, + matched_count: matched.length, + quota_exceeded: quotaExceeded, + _meta: { region: client.region }, + }; + if (truncatedAt !== undefined) out.truncated_at = truncatedAt; + + return withAgentMemoryMeta(client, out, ctx); + }, +}; diff --git a/packages/core/src/index.ts b/packages/core/src/index.ts index cc5415b0..b65fe8ab 100644 --- a/packages/core/src/index.ts +++ b/packages/core/src/index.ts @@ -99,6 +99,7 @@ import { campaignCallSheet } from "./composite/campaign-call-sheet.js"; import { researchLeadById } from "./composite/research-lead-by-id.js"; import { researchLeadByNameFuzzy } from "./composite/research-lead-by-name-fuzzy.js"; import { accountHistory } from "./composite/account-history.js"; +import { scanPortfolioSignals } from "./composite/scan-portfolio-signals.js"; import { recallOrderedTitles } from "./composite/recall-ordered-titles.js"; import { accountStatus } from "./composite/account-status.js"; import { bulkQualifyLeads } from "./composite/bulk-qualify-leads.js"; @@ -163,7 +164,7 @@ export { pullLeads, pullFollowups, followupsMap, tourPlan, listCampaigns, campaignProgression, campaignCallSheet, researchLeadById, researchLeadByNameFuzzy, accountHistory, - recallOrderedTitles, accountStatus, + recallOrderedTitles, accountStatus, scanPortfolioSignals, bulkEnrichStatus, qualifyStatus, importStatus, resolveImportRows, // new composite writes bulkQualifyLeads, enrichTitles, adjustAudience, refinePrompt, @@ -265,6 +266,10 @@ export const compositeReadTools: Tool[] = [ // reprioritize-a-neglected-account workflow (#3630 GAP C) must work in a // default deployment without LEADBAY_MCP_ADVANCED=1. accountHistory, + // Bulk portfolio signal scan — read-only, no quota burn. The single-call + // answer to "which of my leads have signal X" that previously forced a + // per-lead research_lead_by_id loop (issue #3704). + scanPortfolioSignals, recallOrderedTitles, accountStatus, bulkEnrichStatus, diff --git a/packages/core/src/tool-descriptions.generated.ts b/packages/core/src/tool-descriptions.generated.ts index fb6ebfb1..c07fea76 100644 --- a/packages/core/src/tool-descriptions.generated.ts +++ b/packages/core/src/tool-descriptions.generated.ts @@ -2290,9 +2290,9 @@ table. Detail + status priority below. Pull KNOWN leads from the user's Monitor view — the re-engagement entry point. Use when the user asks "what should I follow up on", "leads I haven't contacted", "leads in [city]", "before my trip", or any phrasing implying pre-existing pipeline context. For NEW leads from Discover, use \`leadbay_pull_leads\`. -Backend: wraps \`GET /1.5/monitor?personal=&liked=&filtered=&count=&page=\` plus, when \`set_filter\` is supplied, a preceding \`POST /1.5/monitor/filter\` to persist the filter server-side. The Monitor filter is a single \`FilterItem\` per user — refreshing the page restores it. +Backend: wraps \`GET /1.5/monitor?personal=&liked=&filtered=&count=&page=\` plus, when \`set_filter\` is supplied, a preceding \`POST /1.5/monitor/filter\`. The Monitor filter is a single \`FilterItem\` per user — refreshing restores it. -**Filter mechanism — store-then-apply.** Pass \`set_filter: { criteria: FilterCriterion[] }\` to overwrite the server-stored filter, then the composite re-fetches with \`filtered:true\`. \`FilterCriterion\` is the backend's \`anyOf\` over 10 typed criteria: \`size\`, \`keywords\`, \`sector_ids\`, \`location_ids\`, \`custom_field\`, \`custom_field_comparison\`, \`yc\`, \`liked\`, \`last_action\` (filters by MonitorActionType enum), \`last_action_date\` (with \`last_days\` for "last N days"). +**Filter mechanism — store-then-apply.** Pass \`set_filter: { criteria: FilterCriterion[] }\` to overwrite the server-stored filter, then the composite re-fetches with \`filtered:true\`. \`FilterCriterion\` is the backend's \`anyOf\` over 10 typed criteria: \`size\`, \`keywords\`, \`sector_ids\`, \`location_ids\`, \`custom_field\`(\`_comparison\`), \`yc\`, \`liked\`, \`last_action\` (MonitorActionType enum), \`last_action_date\` (with \`last_days\`). Practical mapping from user phrasing to criterion: @@ -2305,11 +2305,11 @@ Practical mapping from user phrasing to criterion: | "leads 50–200 employees" | \`{type: "size", sizes: [{min: 50, max: 200}]}\` | | "Y Combinator companies" | \`{type: "yc"}\` | -Geo filtering needs \`admin_area_id\` resolution — backend rejects free-text in \`location_ids\`. Pass \`city: ""\` and the composite calls \`/geo/search\` internally, picks the best match, merges its id into \`set_filter\`. Ambiguous matches return \`status: "ambiguous_locations"\` + \`location_ambiguities[]\` — pick an id and re-call with \`city_id\`. For multi-city cases, call \`leadbay_list_locations\` then pass \`set_filter.criteria\` with \`{type: "location_ids", is_excluded: false, locations: [...]}\`. +Geo filtering needs \`admin_area_id\` resolution — backend rejects free-text in \`location_ids\`. Pass \`city: ""\` and the composite calls \`/geo/search\` internally, picks the best match, merges its id into \`set_filter\`. Ambiguous matches return \`status: "ambiguous_locations"\` + \`location_ambiguities[]\` — pick an id and re-call with \`city_id\`. -**Place names go through \`city\`, NEVER \`keywords\`.** This includes any geographic token the user names — cities (\`"Berlin"\`, \`"NYC"\`), states / provinces / regions (\`"Texas"\`, \`"California"\`, \`"Bavaria"\`), countries (\`"France"\`, \`"United States"\`), neighborhoods (\`"Brooklyn"\`, \`"SoHo"\`). The \`/geo/search\` resolver handles all admin levels — level 4 (state) and level 2 (country) resolve just as well as level 5 (city). If you put \`"Texas"\` in \`keywords\` you get a TEXT-MATCH against company descriptions (≈0 hits) instead of a real state filter. If a place name resolves ambiguously, surface the choices to the user — do NOT silently fall back to keyword search or to the unfiltered Monitor view. If \`keywords: ["Texas"]\` returned empty, the next call is \`city: "Texas"\`, not \`keywords: []\`. +**Place names go through \`city\`, NEVER \`keywords\`.** Any geographic token the user names — cities (\`"Berlin"\`), states/regions (\`"Texas"\`, \`"Bavaria"\`), countries (\`"France"\`), neighborhoods (\`"Brooklyn"\`) — resolves via \`/geo/search\` (all admin levels). A place name in \`keywords\` becomes a TEXT-MATCH against company descriptions (≈0 hits), not a real filter. If a place resolves ambiguously, surface the choices — never silently fall back to keyword search or the unfiltered view. -**Pushback exclusion.** Leads with active pushback (\`pushback_status\` set and \`pushback_until > today\`) are excluded from the response. The composite enforces this client-side; \`total_excluded_by_pushback\` in the output reports how many rows were dropped. +**Pushback exclusion.** Leads with active pushback (\`pushback_status\` set, \`pushback_until > today\`) are excluded client-side; \`total_excluded_by_pushback\` reports how many rows were dropped. WHEN TO USE: re-engaging pipeline ("what should I follow up on", "stale leads"), filtering monitored leads by city / sector / recency / action type / liked. The canonical orchestrator is the \`leadbay_followup_check_in\` prompt. @@ -2317,6 +2317,16 @@ WHEN NOT TO USE: for NEW leads — that's \`leadbay_pull_leads\` (Discover). **Anti-confusion guardrail.** Iterating \`pull_leads\` pages looking for \`prospecting_actions_count > 0\` or \`notes_count > 0\` rows is the wrong entry point — the two read different tables. Leads with follow-up history live in \`pull_followups\`. +**SIGNAL HONESTY — never infer signals from freshness.** \`stale_at\`, +\`web_fetch_in_progress\`, \`fetch_at\` are freshness markers, not signal +indicators — signal presence is read ONLY from the actual \`signals[]\` / +\`web_fetch.content\` entries. For "which of my leads have signal X" across a +portfolio, call **\`leadbay_scan_portfolio_signals\`** (bulk-reads cached +signals); don't loop \`leadbay_research_lead_by_id\` per lead or guess from +freshness. A lead with no cached content is \`not_researched\`, not "no match"; +never report a signal verdict for a lead you never read. + + --- ## RENDERING — follow-ups table, status-badge driven @@ -2868,42 +2878,46 @@ Examples that should NOT invoke this tool (sound similar, route elsewhere): --- Tell me everything decision-relevant about a single lead, identified by its -Leadbay UUID. Bundles the lens-scoped lead profile, the AI qualification -answers (the agent's knowledge-base food), the structured web-research signals -(with hot flags + sources), the two-tier contact set (\`enriched\` + \`org\`), the -unified \`recent_activities\` timeline, the engagement counts, and a -\`_meta.has_reachable_contact\` hint that drives NEXT STEPS. Order is -deliberate: qualification first, then signals, then firmographics, then -contacts, then recent activity. - -Scoring has two layers: the basic \`score\` (firmographic, always present, -already decent) and the AI qualification layer (\`ai_agent_lead_score\` + -per-question answers + web_fetch signals). The AI layer is pre-populated for -roughly the top 10 of each daily batch, and on-demand (via -leadbay_bulk_qualify_leads) for anything below that. Combine both layers when -judging a lead. - -The companion tool **leadbay_research_lead_by_name_fuzzy** wraps this one for -the case where the user names a company in prose without a UUID — it -fuzzy-resolves the name against the active lens's wishlist, then delegates -here. Both return the same shape; the fuzzy wrapper just adds -\`_meta.resolved_from\` and \`_meta.match_candidates\` so you can offer -disambiguation. +Leadbay UUID. Bundles the lens-scoped profile, AI qualification answers, +structured web-research signals (hot flags + sources), the two-tier contact set +(\`enriched\` + \`org\`), the unified \`recent_activities\` timeline, engagement +counts, and a \`_meta.has_reachable_contact\` hint that drives NEXT STEPS. Order +is deliberate: qualification, signals, firmographics, contacts, recent activity. + +Scoring has two layers: the basic \`score\` (firmographic, always present) and +the AI qualification layer (\`ai_agent_lead_score\` + per-question answers + +web_fetch signals). The AI layer is pre-populated for roughly the top 10 of +each daily batch, and on-demand (via leadbay_bulk_qualify_leads) below that. +Combine both when judging a lead. + +The companion **leadbay_research_lead_by_name_fuzzy** wraps this one when the +user names a company without a UUID: it fuzzy-resolves against the active +lens's wishlist, then delegates here. Same shape, plus \`_meta.resolved_from\` / +\`_meta.match_candidates\`. WHEN TO USE: when picking up a single lead from leadbay_pull_leads (or any list that exposed a leadId) to decide whether to act on it. WHEN NOT TO USE: across many leads at once — that's -leadbay_pull_leads' job. (This composite supersedes the lower-level -leadbay_get_lead_profile in agent flow; the granular tool stays available for -fine-grained access.) - -**Concurrency note**: this is a composite that reads many sub-resources per -call. Call it **sequentially** or in small batches (≤3 parallel) when -researching multiple leads. Firing 10+ in parallel can saturate the transport -and produce misleading \`"Tool permission stream closed"\` errors that look like -permission failures but are really backpressure. On a transient +leadbay_pull_leads' job (portfolio-wide signal questions go to +leadbay_scan_portfolio_signals; see below). This composite supersedes the +lower-level leadbay_get_lead_profile. + +**SIGNAL HONESTY — never infer signals from freshness.** \`stale_at\`, +\`web_fetch_in_progress\`, \`fetch_at\` are freshness markers, not signal +indicators — signal presence is read ONLY from the actual \`signals[]\` / +\`web_fetch.content\` entries. For "which of my leads have signal X" across a +portfolio, call **\`leadbay_scan_portfolio_signals\`** (bulk-reads cached +signals); don't loop \`leadbay_research_lead_by_id\` per lead or guess from +freshness. A lead with no cached content is \`not_researched\`, not "no match"; +never report a signal verdict for a lead you never read. + + +**Concurrency note**: this composite reads many sub-resources per call. Call +it **sequentially or in small batches (≤3 parallel)**. Firing 10+ in parallel +saturates the transport and produces misleading \`"Tool permission stream +closed"\` errors — that's backpressure, not a permission failure. On a transient stream/timeout failure, retry the same lead once before moving on. --- @@ -3276,6 +3290,181 @@ Below the table, a one-liner: \`"Ready: K rows · Ambiguous: A rows · Unmatched `; // endregion: leadbay_resolve_import_rows +// region: leadbay_scan_portfolio_signals +export const leadbay_scan_portfolio_signals: string = `## WHEN TO USE + +Trigger phrases: "which of my leads ", "find leads that ", "scan my portfolio for ", "identify all the ones that since ", "who in Monitor has a signal", "build a campaign from leads with ". + +**Memory:** recall + capture via \`leadbay_agent_memory_*\` tools. + +Do NOT use for: "research one named company" → \`leadbay_research_lead_by_name_fuzzy\`; "everything about lead " → \`leadbay_research_lead_by_id\`; "qualify my next N leads (they aren't researched yet)" → \`leadbay_bulk_qualify_leads\`; "just list my follow-ups" → \`leadbay_pull_followups\`. + +Prefer when: user wants to FILTER a known portfolio by a web-research signal in bulk — pass \`query\`, optionally \`since\`, \`city\`/\`set_filter\`, or \`leadIds\` + +Examples that SHOULD invoke this tool: +- "Which of my leads acquired a company since 2025?" +- "Scan my Lyon portfolio for funding signals." +- "Find everyone in Monitor who changed CEO and build a campaign." + +Examples that should NOT invoke this tool (sound similar, route elsewhere): +- "Look up Acme Corp for me." +- "Show me my follow-ups." +- "Qualify my next 10 leads." + +## RENDER (quick) + +Cohort grouped by lead: one block per matched lead (name · location + +its matched signal entries, hot first, source-linked). Open with +"N match (M scanned)"; ALWAYS close with an honesty footer — +"scanned N · matched M · K not yet researched". Never present +not_researched leads as "no signal". Full layout below. + +--- + +Scan a known portfolio for a specific web-research signal in one call. This is +the bulk, read-only answer to "which of my leads have signal X" — the question +that otherwise forces a per-lead \`leadbay_research_lead_by_id\` loop (one full +profile call per lead, slow and quota-heavy). + +**Reads CACHED signals only — does not trigger new research.** For each lead in +scope it reads \`GET /leads/{id}/web_fetch\` (the already-computed web-research +signals) and filters the entries against \`query\`. It issues NO web_fetch POST, +so it does not consume AI qualification credits and does not re-crawl. Leads +that have no cached content (never qualified, or still in progress) are +reported in \`not_researched\` — they are **NOT** silently treated as "no +match". Qualify them with \`leadbay_bulk_qualify_leads\`, then re-scan. + +**Scope.** Pass \`leadIds\` for an explicit cohort, or omit it to scan the +Monitor portfolio. Narrow the Monitor scope with \`city\` / \`set_filter\` exactly +as \`leadbay_pull_followups\` does (store-then-apply server-side filter). The +scan is bounded by \`max_leads\` (default 200, hard cap 300); when the portfolio +is larger, \`truncated_at\` is set and coverage is partial — say so. + +**Query.** \`query\` is matched case- and accent-insensitively against each +signal entry's description, source, and section label. Comma- or +space-separated terms are OR'd ("M&A, acquisition, racheté" matches any). Use +\`since\` (ISO date) to keep only entries dated on/after it — entries with no +date are kept (a missing date is not evidence the event is old). + +**Result is campaign-ready.** \`matched[]\` carries \`lead_id\`, \`name\`, +\`location\`, and the matching \`matched_signals[]\` (section + hot + source + +date + description). Feed the matched \`lead_id\`s straight into +\`leadbay_add_leads_to_campaign\` / \`leadbay_create_campaign\`. + +On a 429 mid-scan, partial \`matched\` is returned with \`quota_exceeded: true\` — +offer the user wait-for-reset OR a top-up link (both unblock; a top-up clears +the throttle immediately). + +**SIGNAL HONESTY — never infer signals from freshness.** \`stale_at\`, +\`web_fetch_in_progress\`, \`fetch_at\` are freshness markers, not signal +indicators — signal presence is read ONLY from the actual \`signals[]\` / +\`web_fetch.content\` entries. For "which of my leads have signal X" across a +portfolio, call **\`leadbay_scan_portfolio_signals\`** (bulk-reads cached +signals); don't loop \`leadbay_research_lead_by_id\` per lead or guess from +freshness. A lead with no cached content is \`not_researched\`, not "no match"; +never report a signal verdict for a lead you never read. + + +WHEN TO USE: when the user wants to filter a known +portfolio by a web-research signal across many leads at once — discovering a +cohort to act on, not inspecting a single lead. + +WHEN NOT TO USE: for a single named company +(leadbay_research_lead_by_name_fuzzy) or one lead by UUID +(leadbay_research_lead_by_id); to qualify leads that have no signals yet +(leadbay_bulk_qualify_leads); or to just list follow-ups with no signal filter +(leadbay_pull_followups). + +--- + +## RENDERING — bulk signal-scan results + +The output is a cohort, grouped by lead. Lead with the matches, end with an +honesty footer — never hide what wasn't scanned. + +### Matched leads + +Open with a one-line headline: \`**N leads match ""** (M scanned).\` + +Then one block per \`matched[]\` lead, ordered with \`hot\` matches first. Emit +each as a host-parseable per-lead block so the chat host's place-card +auto-detector can render it (per the repo "feed the address auto-detector" +convention): + +\`\`\` +### · + + +- ** ** — <🔥 if hot> ([source](), ) +\`\`\` + +- **Bold** the description of \`hot: true\` entries; leave cold entries plain. +- Render \`source\` as a markdown link \`([source](url), date)\`; omit the date + when null, omit the link when \`source\` is empty. +- Cap to the 3 strongest signals per lead (hot first, then by date desc); if a + lead has more, end its block with \`_+K more signals_\`. +- When \`name\` is null (the scan was scoped by \`leadIds\` and the read failed to + carry firmographics), fall back to \`### Lead \` — but prefer to enrich + the name via the matched lead's own data when available. + +### Honesty footer (ALWAYS print) + +A single italic line summarising coverage: + +\`_Scanned N · matched M · K had no cached signals (not yet researched)._\` + +- When \`not_researched\` is non-empty, this is load-bearing: state plainly that + those K leads were NOT searched and were NOT counted as "no match". Offer to + qualify them and re-scan (see NEXT STEPS). +- When \`truncated_at\` is set, add: \`_Coverage partial — only the first + leads were scanned; narrow the scope or raise max_leads._\` +- When \`quota_exceeded\` is true, add the wait-or-top-up offer. + +**Hide:** raw \`lead_id\` in prose (use it only for the campaign call), \`_meta\`, +empty arrays, any freshness field. NEVER present \`not_researched\` leads as +"no signal found". + + +--- + +## NEXT STEPS — after the signal scan + +**ALWAYS render NEXT STEPS via your host's next-step widget.** Use whichever is in your tool set — the NAME and SCHEMA differ: **\`ask_user_input_v0\`** (Claude chat / ChatGPT) takes plain-string options with \`type:"single_select"\`; **\`AskUserQuestion\`** (Claude cowork / Claude Code) takes object options \`{label, description}\` plus a required short \`header\` (≤12 chars) and \`multiSelect\`, NO \`type\` field, and never add an "Other" option (the host adds it). Match the schema to the tool you actually have — the wrong schema fails silently and you fall back to prose. Prose bullets are the fallback ONLY when NEITHER widget exists. Any turn that would end with a choice must be the widget — the widget IS the question. + +**If the tool result carries a \`next_steps\` object, that is the source of truth — use it directly.** Each option has a short \`.label\` (≤5 words) and a full \`.description\`. Map \`next_steps.options[]\` into your host widget VERBATIM and in order: for \`AskUserQuestion\` (cowork / Claude Code) pass each as \`{label, description}\`; for \`ask_user_input_v0\` (Claude chat / ChatGPT, string options only) pass each option's \`.description\` as the string (it's the full sentence). Do NOT reword, reorder, drop, or prose-ify them — they're built deterministically by the server so the offer (incl. the artifact option at position 0) fires every time. Fall back to the table below only when there is NO \`next_steps\` field. + +**One exception — skip the widget** when the user's original message contained a complete sequential instruction chain ("show me X and then do Y") AND all stated steps have been completed. In that case, end with STOP directly — the user stated their full plan and does not need a "what next?" prompt. +- Skip example: "Show me today's leads and then research the top one for me." → after research completes, emit STOP without the widget. +- Do NOT skip for: plain requests ("show me today's leads", "run my check-in"), recurring-language requests ("I do this every day"), or requests where only one action was stated. + +Pick 2–4 rows from the (Observation, Suggest, Calls) table below most relevant to the response, then call your host's widget with ITS schema (per the schema rules above — wrong schema fails silently): +- \`ask_user_input_v0\`: \`{questions:[{question,type:"single_select",options:["",""]}]}\` +- \`AskUserQuestion\`: \`{questions:[{question,header:"Next step",multiSelect:false,options:[{label:"<≤5 words>",description:""}]}]}\` + +User picks → call the matching \`Calls\` tool. Constraints: 2–4 mutually-exclusive options, AskUserQuestion labels ≤5 words (full text in \`description\`), max 3 questions. Table stays internal; never recite it. + +--- + + + +The scan exists to BUILD A COHORT, not just to list. The default next move is +almost always "turn the matched leads into a campaign." + +| Observation | Suggest | Calls | +|---------------------------------------------------|--------------------------------------------------------------|----------------------------------------------------------------------------------------| +| \`matched\` non-empty (top of menu) | "Build a campaign from the N matched leads" | leadbay_create_campaign / leadbay_add_leads_to_campaign(matched lead_ids) | +| \`not_researched\` non-empty | "K leads aren't researched yet — qualify them, then re-scan" | leadbay_bulk_qualify_leads(not_researched lead_ids) → re-run leadbay_scan_portfolio_signals | +| Zero matches but leads were researched | "Widen the query (synonyms) or relax \`since\`" | leadbay_scan_portfolio_signals(query: "", since: omit-or-earlier) | +| \`truncated_at\` set | "Scan only covered N — narrow scope or raise the cap" | leadbay_scan_portfolio_signals({city / set_filter}) or raise \`max_leads\` | +| One standout matched lead | "Open that lead's full brief" | leadbay_research_lead_by_id(leadId) | +| \`quota_exceeded\` | "Wait for reset OR top up to finish the scan" | leadbay_create_topup_link | + +NEVER report leads in \`not_researched\` as if they had no matching signal — they +were never read. Distinguish "no signal X found" (researched, no match) from +"not yet researched" (no data to search) every time. +`; +// endregion: leadbay_scan_portfolio_signals + // region: leadbay_seed_candidates export const leadbay_seed_candidates: string = `## WHEN TO USE @@ -3674,6 +3863,7 @@ export const TOOL_DESCRIPTIONS = { leadbay_research_lead_by_id, leadbay_research_lead_by_name_fuzzy, leadbay_resolve_import_rows, + leadbay_scan_portfolio_signals, leadbay_seed_candidates, leadbay_select_leads, leadbay_set_active_lens, diff --git a/packages/core/test/unit/composite/scan-portfolio-signals.test.ts b/packages/core/test/unit/composite/scan-portfolio-signals.test.ts new file mode 100644 index 00000000..2e329908 --- /dev/null +++ b/packages/core/test/unit/composite/scan-portfolio-signals.test.ts @@ -0,0 +1,378 @@ +/** + * Unit tests for leadbay_scan_portfolio_signals — the bulk, read-only + * portfolio signal scan (issue #3704). Verifies: it reads CACHED signals + * (no web_fetch POST), filters by query + since, separates "no match" from + * "not researched", folds diacritics/case, caps the fan-out, and survives a + * 429 mid-scan with partial results. + */ + +import { describe, it, expect, beforeEach, vi } from "vitest"; +import { + mockHttp, + resetHttpMock, + httpsMockFactory, + getHttpRequests, +} from "../../harness.js"; + +vi.mock("node:https", () => httpsMockFactory()); + +import { LeadbayClient } from "../../../src/client.js"; +import { scanPortfolioSignals } from "../../../src/composite/scan-portfolio-signals.js"; + +const BASE = "https://api-us.leadbay.app"; +const newClient = () => new LeadbayClient(BASE, "u.tok", "us"); + +// A web_fetch payload with the given emoji-section → entries shape. +function webFetch(leadId: string, content: any, inProgress = false) { + return { + method: "GET" as const, + path: `/1.5/leads/${leadId}/web_fetch`, + status: 200, + body: { lead_id: leadId, content, fetch_at: "2025-06-01", in_progress: inProgress }, + }; +} + +beforeEach(() => resetHttpMock()); + +describe("leadbay_scan_portfolio_signals", () => { + it("happy path — returns only leads whose signals match the query, with entries quoted", async () => { + mockHttp([ + webFetch("lead-1", { + "📈 business signals": [ + { description: "Acme acquired BetaCorp in a $40M deal", source: "techcrunch.com", date: "2025-03-01", hot: true }, + { description: "Hiring 20 engineers", source: "linkedin.com", date: "2025-02-01" }, + ], + }), + webFetch("lead-2", { + "📈 business signals": [ + { description: "Opened a new office in Lyon", source: "lemonde.fr", date: "2025-04-01" }, + ], + }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired, M&A", + leadIds: ["lead-1", "lead-2"], + }); + + expect(out.matched).toHaveLength(1); + expect(out.matched[0].lead_id).toBe("lead-1"); + expect(out.matched[0].matched_signals).toHaveLength(1); + expect(out.matched[0].matched_signals[0].description).toContain("acquired BetaCorp"); + expect(out.matched[0].matched_signals[0].hot).toBe(true); + expect(out.matched_count).toBe(1); + expect(out.scanned_count).toBe(2); + expect(out.not_researched).toHaveLength(0); + + // Read-only: NO web_fetch POST was issued. + const posts = getHttpRequests().filter((r) => r.method === "POST"); + expect(posts).toHaveLength(0); + }); + + it("separates 'not researched' (null/in-progress content) from 'no match'", async () => { + mockHttp([ + webFetch("lead-1", { + "📈 business signals": [{ description: "raised a Series B", source: "x.com", date: "2025-05-01" }], + }), + webFetch("lead-2", null), // never researched + webFetch("lead-3", { "📈 business signals": [{ description: "x" }] }, true), // still fetching + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "funding, Series", + leadIds: ["lead-1", "lead-2", "lead-3"], + }); + + expect(out.matched.map((m: any) => m.lead_id)).toEqual(["lead-1"]); + // lead-2 (null) and lead-3 (in_progress) → not_researched, NOT silently dropped. + expect(out.not_researched.map((n: any) => n.lead_id).sort()).toEqual(["lead-2", "lead-3"]); + expect(out.scanned_count).toBe(3); + }); + + it("since filter — entries dated before `since` are excluded; undated entries kept", async () => { + mockHttp([ + webFetch("lead-1", { + "📈 business signals": [ + { description: "acquired OldCo", source: "s", date: "2024-08-01" }, // before since + { description: "acquired NewCo", source: "s", date: "2025-02-01" }, // after since + { description: "acquired UndatedCo", source: "s" }, // no date → kept + ], + }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired", + leadIds: ["lead-1"], + since: "2025-01-01", + }); + + const descs = out.matched[0].matched_signals.map((s: any) => s.description); + expect(descs).toContain("acquired NewCo"); + expect(descs).toContain("acquired UndatedCo"); + expect(descs).not.toContain("acquired OldCo"); + }); + + it("diacritic- and case-insensitive matching — accented query 'racheté' matches plain 'rachete', and 'M&A' matches 'm&a'", async () => { + mockHttp([ + webFetch("lead-1", { + // entry has NO accent + different case; query carries the accent + case. + "📈 signals": [{ description: "L'entreprise a RACHETE un concurrent", source: "lesechos.fr", date: "2025-03-01" }], + }), + webFetch("lead-2", { + "📈 signals": [{ description: "Completed an M&A transaction", source: "ft.com", date: "2025-03-01" }], + }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + // "racheté" folds to "rachete" (matches lead-1 regardless of accent/case); + // "m&a" matches lead-2's "M&A" case-insensitively. + query: "racheté, m&a", + leadIds: ["lead-1", "lead-2"], + }); + + expect(out.matched.map((m: any) => m.lead_id).sort()).toEqual(["lead-1", "lead-2"]); + }); + + it("no matches — returns empty matched[], scanned_count correct, no throw", async () => { + mockHttp([ + webFetch("lead-1", { "📈 signals": [{ description: "nothing relevant", source: "s" }] }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquisition", + leadIds: ["lead-1"], + }); + + expect(out.matched).toHaveLength(0); + expect(out.matched_count).toBe(0); + expect(out.scanned_count).toBe(1); + expect(out.not_researched).toHaveLength(0); + }); + + it("429 mid-scan — partial matched preserved, quota_exceeded true", async () => { + mockHttp([ + webFetch("lead-1", { + "📈 signals": [{ description: "acquired a startup", source: "s", date: "2025-03-01" }], + }), + { + method: "GET", + path: "/1.5/leads/lead-2/web_fetch", + status: 429, + body: { code: "QUOTA_EXCEEDED" }, + }, + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired", + leadIds: ["lead-1", "lead-2"], + }); + + expect(out.quota_exceeded).toBe(true); + expect(out.matched.map((m: any) => m.lead_id)).toEqual(["lead-1"]); + expect(out.matched_count).toBe(1); + // lead-2 failed to read → surfaced as not_researched, never silently + // dropped nor falsely "no match" (issue #3704 honesty invariant). + expect(out.not_researched.map((n: any) => n.lead_id)).toEqual(["lead-2"]); + // scanned_count = matched + non-matching + not_researched holds. + expect(out.scanned_count).toBe(2); + }); + + it("non-quota read failure (404) — lead surfaces as not_researched, not silently dropped", async () => { + mockHttp([ + webFetch("lead-1", { + "📈 signals": [{ description: "acquired a startup", source: "s", date: "2025-03-01" }], + }), + { + method: "GET", + path: "/1.5/leads/lead-2/web_fetch", + status: 404, + body: { code: "NOT_FOUND" }, + }, + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired", + leadIds: ["lead-1", "lead-2"], + }); + + // A 404 is not a quota wall — but the lead still couldn't be read, so it + // must land in not_researched (honest coverage), and scanned_count must + // still account for it. + expect(out.quota_exceeded).toBe(false); + expect(out.matched.map((m: any) => m.lead_id)).toEqual(["lead-1"]); + expect(out.not_researched.map((n: any) => n.lead_id)).toEqual(["lead-2"]); + expect(out.scanned_count).toBe(2); + }); + + it("max_leads cap — truncated_at is set when leadIds exceed the cap", async () => { + mockHttp([ + webFetch("lead-1", { "📈 signals": [{ description: "acquired co", source: "s", date: "2025-03-01" }] }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired", + leadIds: ["lead-1", "lead-2", "lead-3"], + max_leads: 1, + }); + + expect(out.truncated_at).toBe(1); + expect(out.scanned_count).toBe(1); + }); + + it("Monitor scope (city) — resolves geo, filters, paginates /monitor, then bulk-reads web_fetch", async () => { + mockHttp([ + // 1. geo resolve — exact-name match on "Lyon" short-circuits ambiguity. + { + method: "GET", + path: "/1.5/geo/search?q=Lyon", + status: 200, + body: { results: [{ id: "geo-lyon", name: "Lyon", country: "FR", level: 5 }] }, + }, + // 2. store the filter (location_ids merged from the resolved geo id). + { method: "POST", path: "/1.5/monitor/filter", status: 200, body: {} }, + // 3. one short page of the portfolio — carries id/name/location. + { + method: "GET", + path: /^\/1\.5\/monitor\?/, + status: 200, + body: { + items: [ + { id: "lead-1", name: "Acme SARL", location: { city: "Lyon", state: "ARA" } }, + { id: "lead-2", name: "Beta SA", location: "Lyon, ARA" }, + ], + pagination: { pages: 1 }, + }, + }, + // 4. per-lead cached signal reads. + webFetch("lead-1", { + "📈 business signals": [ + { description: "Acme racheté par un groupe US", source: "lesechos.fr", date: "2025-03-01", hot: true }, + ], + }), + webFetch("lead-2", { + "📈 business signals": [{ description: "embauche 10 personnes", source: "x", date: "2025-02-01" }], + }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "racheté, acquisition", + city: "Lyon", + }); + + expect(out.matched.map((m: any) => m.lead_id)).toEqual(["lead-1"]); + // name + location carried through from the Monitor page. + expect(out.matched[0].name).toBe("Acme SARL"); + expect(out.matched[0].location).toBe("Lyon, ARA"); + expect(out.scanned_count).toBe(2); + + const reqs = getHttpRequests(); + // The filter was stored (POST) and /monitor was queried with filtered=true. + expect(reqs.some((r) => r.method === "POST" && r.path === "/1.5/monitor/filter")).toBe(true); + const monitorReq = reqs.find((r) => r.method === "GET" && r.path.startsWith("/1.5/monitor?")); + expect(monitorReq?.path).toContain("filtered=true"); + }); + + it("429 while paging /monitor — sets quota_exceeded so the agent reports 'incomplete', not 'no matches'", async () => { + mockHttp([ + { + method: "GET", + path: "/1.5/geo/search?q=Lyon", + status: 200, + body: { results: [{ id: "geo-lyon", name: "Lyon", country: "FR", level: 5 }] }, + }, + { method: "POST", path: "/1.5/monitor/filter", status: 200, body: {} }, + // The portfolio enumeration itself hits the quota wall. + { + method: "GET", + path: /^\/1\.5\/monitor\?/, + status: 429, + body: { code: "QUOTA_EXCEEDED" }, + }, + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired", + city: "Lyon", + }); + + // Could not enumerate the portfolio → honest partial coverage, NOT a + // confident empty result. + expect(out.quota_exceeded).toBe(true); + expect(out.matched).toHaveLength(0); + expect(out.scanned_count).toBe(0); + }); + + it("filter POST fails — falls back to an UNfiltered scan (filtered=false), never trusts a stale server-side filter", async () => { + mockHttp([ + // Storing the filter fails (server error). + { method: "POST", path: "/1.5/monitor/filter", status: 500, body: { code: "SERVER_ERROR" } }, + // The /monitor read must go out UNfiltered to avoid a stale cohort. + { + method: "GET", + path: /^\/1\.5\/monitor\?/, + status: 200, + body: { items: [{ id: "lead-1", name: "Acme", location: "Paris" }], pagination: { pages: 1 } }, + }, + webFetch("lead-1", { + "📈 signals": [{ description: "acquired a rival", source: "s", date: "2025-03-01" }], + }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired", + set_filter: { criteria: [{ type: "sector_ids", is_excluded: false, sectors: ["x"] }] } as any, + }); + + expect(out.matched.map((m: any) => m.lead_id)).toEqual(["lead-1"]); + const monitorReq = getHttpRequests().find( + (r) => r.method === "GET" && r.path.startsWith("/1.5/monitor?") + ); + // The store failed, so the read must NOT claim filtered=true. + expect(monitorReq?.path).toContain("filtered=false"); + }); + + it("ambiguous city — returns status:'ambiguous_locations' and issues no /monitor or web_fetch calls", async () => { + mockHttp([ + // Two close-scoring prefix matches, neither an exact-name win → ambiguous. + { + method: "GET", + path: "/1.5/geo/search?q=Springfield", + status: 200, + body: { + results: [ + { id: "geo-il", name: "Springfield township", country: "US", level: 8 }, + { id: "geo-mo", name: "Springfield village", country: "US", level: 8 }, + ], + }, + }, + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: "acquired", + city: "Springfield", + }); + + expect(out.status).toBe("ambiguous_locations"); + expect(out.location_ambiguities.length).toBeGreaterThan(0); + expect(out.scanned_count).toBe(0); + expect(out.matched).toHaveLength(0); + + const reqs = getHttpRequests(); + expect(reqs.some((r) => r.path.includes("/monitor"))).toBe(false); + expect(reqs.some((r) => r.path.includes("/web_fetch"))).toBe(false); + }); + + it("empty/whitespace query — matches nothing (no false positives)", async () => { + mockHttp([ + webFetch("lead-1", { "📈 signals": [{ description: "acquired co", source: "s" }] }), + ]); + + const out: any = await scanPortfolioSignals.execute(newClient(), { + query: " ", + leadIds: ["lead-1"], + }); + + expect(out.matched).toHaveLength(0); + expect(out.scanned_count).toBe(1); + }); +}); diff --git a/packages/mcp/CHANGELOG.md b/packages/mcp/CHANGELOG.md index e9d18e13..9fbacc9a 100644 --- a/packages/mcp/CHANGELOG.md +++ b/packages/mcp/CHANGELOG.md @@ -1,5 +1,11 @@ # Changelog — @leadbay/mcp +## 0.19.1 — 2026-06-09 + +- **New tool `leadbay_scan_portfolio_signals`**: read-only bulk scan of a Monitor portfolio (or an explicit lead-id list) for a web-research signal. Ask "which of my leads have an M&A / funding / hiring signal since 2025" and get the matched cohort back in one call — a `GET`-only fan-out over cached `web_fetch` signals (no per-lead research loop, no AI-qualification quota burn), with a case- and accent-folded query and optional `since` date. The matched cohort is campaign-ready (feeds straight into `leadbay_add_leads_to_campaign`). +- **Signal-honesty guardrail**: the scan separates `not_researched[]` (no cached content) from "no match", so the agent can never claim coverage for leads it never read. Reinforced in `leadbay_pull_followups`, `leadbay_research_lead_by_id`, and the `followup_check_in` prompt: freshness fields (`stale_at`, `web_fetch_in_progress`, `fetch_at`) are not signal indicators, and portfolio-wide signal questions route to the bulk tool. Every error path stays honest — a 429 while paging the portfolio, a non-quota read failure, and a failed filter-store all surface partial coverage rather than reporting a confident empty result. +- **Agent-side gap-fill in the follow-up check-in**: PHASE 3b turns the coverage gap into a refinement loop — the agent names the gap, runs a targeted live web pass on only the `not_researched` / thin-signal leads, and folds findings back in clearly labelled as agent-sourced (not Leadbay-verified), with `leadbay_bulk_qualify_leads` offered as the durable path that writes the signal into the portfolio. + ## 0.19.0 — 2026-06-09 NEXT STEPS, artifact, and scheduled-task offers now fire reliably as host widgets across Claude chat, Claude cowork/Desktop, and ChatGPT. diff --git a/packages/mcp/package.json b/packages/mcp/package.json index cffa2944..297d59a0 100644 --- a/packages/mcp/package.json +++ b/packages/mcp/package.json @@ -1,6 +1,6 @@ { "name": "@leadbay/mcp", - "version": "0.19.0", + "version": "0.19.1", "mcpName": "io.github.leadbay/leadbay-mcp", "description": "Model Context Protocol (MCP) server for Leadbay — AI lead discovery, qualification, and enrichment for Claude Desktop, Cursor, and Claude Code.", "type": "module", diff --git a/packages/mcp/server.json b/packages/mcp/server.json index 52bc3e9f..92485112 100644 --- a/packages/mcp/server.json +++ b/packages/mcp/server.json @@ -3,7 +3,7 @@ "name": "io.github.leadbay/leadbay-mcp", "title": "Leadbay", "description": "AI lead discovery, qualification, and outreach prep on your Leadbay account.", - "version": "0.19.0", + "version": "0.19.1", "repository": { "url": "https://github.com/leadbay/leadclaw", "source": "github", @@ -15,7 +15,7 @@ "registryType": "npm", "registryBaseUrl": "https://registry.npmjs.org", "identifier": "@leadbay/mcp", - "version": "0.19.0", + "version": "0.19.1", "transport": { "type": "stdio" }, diff --git a/packages/mcp/src/prompts.generated.ts b/packages/mcp/src/prompts.generated.ts index aab030ff..0d5e733a 100644 --- a/packages/mcp/src/prompts.generated.ts +++ b/packages/mcp/src/prompts.generated.ts @@ -423,6 +423,29 @@ Unlike \`leadbay_daily_check_in\` which deep-dives on every promising lead in Ph When the user picks a row, call \`leadbay_research_lead_by_id\` on that single lead (or \`leadbay_research_lead_by_name_fuzzy\` if they only have the name) and offer to \`leadbay_prepare_outreach\` once they say "let's reach out". +# PHASE 3b — BULK SIGNAL SCAN (when the user asks "which of these have signal X") + +If the user wants to filter the whole portfolio by a web-research signal — "which of my leads acquired a company since 2025", "find everyone with a funding signal", "who changed CEO" — do NOT loop \`leadbay_research_lead_by_id\` per row, and do NOT guess from freshness fields. Call \`leadbay_scan_portfolio_signals({query, since?})\` once: it bulk-reads the cached signals across the portfolio and returns only the matches, campaign-ready. Offer to build a campaign from the matches. + +**Close the gap — don't just report it.** The scan returns \`not_researched[]\` (leads with no cached Leadbay signal) and may surface matches whose cached signal is thin or undated. These are the leads where the answer is genuinely unknown, not absent. Rather than stopping at "K aren't researched yet", offer to fill the gap and refine: + +1. **Name the gap precisely** — "N of your M leads matched; K have no cached signal and J more have only a thin/undated mention, so I can't yet confirm signal X for those." +2. **Run a targeted live pass** — if you have web-search tools, research the specific \`not_researched\` / thin-signal leads for the exact signal the user asked about (company name + the query terms, e.g. " acquisition 2025"). Do this only for the gap leads, not the whole portfolio — the cached matches are already answered. +3. **Fold the findings back in, clearly labelled** — present live results as **agent-sourced (not Leadbay-verified)**, in a section separate from the cached \`matched\` cohort, and cite the source URL/date you found. Never silently merge a web-found signal into the campaign-ready cohort as if Leadbay had verified it. +4. **Offer the durable path too** — for gap leads worth persisting, offer \`leadbay_bulk_qualify_leads\` so Leadbay runs its own \`web_fetch\` and the signal lands in the portfolio's cached \`signals[]\` on the next scan. + +This keeps the honesty guarantee intact (Leadbay's cached \`signals[]\` stay the source of truth) while still answering the user's question for the leads Leadbay hasn't researched yet. + +**SIGNAL HONESTY — never infer signals from freshness.** \`stale_at\`, +\`web_fetch_in_progress\`, \`fetch_at\` are freshness markers, not signal +indicators — signal presence is read ONLY from the actual \`signals[]\` / +\`web_fetch.content\` entries. For "which of my leads have signal X" across a +portfolio, call **\`leadbay_scan_portfolio_signals\`** (bulk-reads cached +signals); don't loop \`leadbay_research_lead_by_id\` per lead or guess from +freshness. A lead with no cached content is \`not_researched\`, not "no match"; +never report a signal verdict for a lead you never read. + + # CROSS-MODE PIVOT Below the table, offer the cross-mode pivot in one short line so the user can redirect if you guessed wrong on entry-point routing: "Want to see NEW leads from your wishlist instead?" — that routes back to \`leadbay_daily_check_in\` (Discovery via \`leadbay_pull_leads\`). diff --git a/packages/mcp/test/audit/routing-block.test.ts b/packages/mcp/test/audit/routing-block.test.ts index 30ce651d..0cf8a3a2 100644 --- a/packages/mcp/test/audit/routing-block.test.ts +++ b/packages/mcp/test/audit/routing-block.test.ts @@ -44,6 +44,7 @@ const TOOLS_WITH_ROUTING = new Set([ "leadbay_report_friction", "leadbay_research_lead_by_id", "leadbay_research_lead_by_name_fuzzy", + "leadbay_scan_portfolio_signals", "leadbay_tour_plan", // leadbay_seed_candidates is INTERNAL scaffolding for leadbay_extend_lens — // users never invoke it directly. Routing frontmatter still exists to help diff --git a/packages/mcp/test/eval/scenarios/scan-portfolio-signals/README.md b/packages/mcp/test/eval/scenarios/scan-portfolio-signals/README.md new file mode 100644 index 00000000..816500fa --- /dev/null +++ b/packages/mcp/test/eval/scenarios/scan-portfolio-signals/README.md @@ -0,0 +1,34 @@ +# Eval scenarios — `leadbay_scan_portfolio_signals` + +Two scenarios guarding the issue #3704 fix. Both are authored to the scenario +shape in `../../README.md` (§"Adding a scenario") and are fixture-complete — +they run as soon as the scenario-execution glue (`helpers/run-eval.ts`, +`setupScenarioFixtures`, `runScenarioEval`, `vitest.eval.config.ts`) lands. That +glue does not exist on this branch yet, so there is intentionally **no +`prompts/*.eval.ts` wiring file** — adding one would import a module that +doesn't exist and break the build. Wire them up like this once the runner is in: + +```ts +// prompts/leadbay_followup_check_in.eval.ts +import { runScenarioEval, setupScenarioFixtures } from "../helpers/run-eval.js"; +import { SCENARIO as FINDS_MA } from "../scenarios/scan-portfolio-signals/finds-ma-cohort.scenario.js"; +import { SCENARIO as HONEST } from "../scenarios/scan-portfolio-signals/honest-about-unresearched.scenario.js"; + +for (const s of [FINDS_MA, HONEST]) { + describe(`eval: ${s.prompt} — ${s.name}`, () => { + setupScenarioFixtures(s); + it(s.name, async () => { await runScenarioEval({ scenario: s }); }); + }); +} +``` + +| Scenario | Failure mode it catches | +|---|---| +| `finds-ma-cohort` | **Underdeliver.** Portfolio has post-2025 M&A signals; the agent must answer via ONE `leadbay_scan_portfolio_signals` call and surface the matched cohort — not loop `leadbay_research_lead_by_id` per lead and not give up (JM's original failure). | +| `honest-about-unresearched` | **Fabrication / `stale_at` confusion.** Some leads have no cached signals; the agent must report them as *not yet researched* (the scan's `not_researched` bucket), never as "no M&A signal", and must not fabricate a verdict (milstan's diagnosis). `no_fabrication` must score 5. | + +Both were validated against the live US test account (`SnapLock Industries`) +during development: `leadbay_scan_portfolio_signals` correctly returned real +M&A matches (e.g. "QUEST DRAPE, LLC — acquired Drape Kings in March 2025", +hot, sourced, dated) across a 60-lead scan in ~2.3s with zero per-lead +research loops. diff --git a/packages/mcp/test/eval/scenarios/scan-portfolio-signals/finds-ma-cohort.scenario.ts b/packages/mcp/test/eval/scenarios/scan-portfolio-signals/finds-ma-cohort.scenario.ts new file mode 100644 index 00000000..5c2f5564 --- /dev/null +++ b/packages/mcp/test/eval/scenarios/scan-portfolio-signals/finds-ma-cohort.scenario.ts @@ -0,0 +1,100 @@ +// Eval scenario — UNDERDELIVER guard for issue #3704. +// +// JM built a 497-lead portfolio and asked "which of these acquired a company +// since 2025". The agent had no bulk path, looped research_lead_by_id ~60 +// times, then gave up. This scenario asserts the FIX: when the portfolio has +// matching M&A signals, the agent must reach for leadbay_scan_portfolio_signals +// (ONE call) and surface the matched cohort — NOT loop research_lead_by_id per +// lead and NOT abandon the task. +// +// Authored to the README scenario shape (test/eval/README.md). Runs once the +// scenario-execution glue (run-eval.ts / setupScenarioFixtures) lands; the +// fixtures + mission are runner-ready today. + +const P = (path: string) => `/1.5${path}`; // LeadbayClient prepends /1.5 + +// A small Monitor portfolio: two leads with a clear post-2025 M&A signal, one +// without. The agent should scan all three and return exactly the two matches. +const MONITOR_LEADS = [ + { + id: "lead-ma-1", + name: "QUEST DRAPE, LLC", + score: 78, + location: { city: "Frisco", state: "Texas", country: "US", full: "Frisco, Texas, United States" }, + recommended_contact: null, + org_contacts: [], + pushback_status: null, + }, + { + id: "lead-ma-2", + name: "ACME EVENTS GROUP", + score: 71, + location: { city: "Austin", state: "Texas", country: "US", full: "Austin, Texas, United States" }, + recommended_contact: null, + org_contacts: [], + pushback_status: null, + }, + { + id: "lead-nomatch-3", + name: "STILLWATER RENTALS INC.", + score: 64, + location: { city: "Tulsa", state: "Oklahoma", country: "US", full: "Tulsa, Oklahoma, United States" }, + recommended_contact: null, + org_contacts: [], + pushback_status: null, + }, +]; + +const wf = (leadId, content) => ({ + method: "GET", + path: P(`/leads/${leadId}/web_fetch`), + status: 200, + body: { lead_id: leadId, in_progress: false, fetch_at: "2025-06-01T00:00:00Z", content }, +}); + +export const SCENARIO = { + name: "scan-finds-ma-cohort", + prompt: "leadbay_followup_check_in", + tier: "gate", + args: {}, + backendFixtures: [ + // Monitor filter read + portfolio page (followup_check_in pulls Monitor first). + { method: "GET", path: P("/monitor/filter"), status: 200, body: { criteria: [] } }, + { + method: "GET", + path: /\/1\.5\/monitor\?/, + status: 200, + body: { items: MONITOR_LEADS, pagination: { page: 0, pages: 1, total: 3 } }, + }, + // Cached signals per lead — the scan reads these (GET only, no POST). + wf("lead-ma-1", { + "📈 business signals": [ + { description: "Acquired Drape Kings in March 2025 to extend product offerings.", source: "Dealroom", date: "2025-03-01", hot: true }, + ], + }), + wf("lead-ma-2", { + "📈 business signals": [ + { description: "Closed acquisition of a regional competitor in Q1 2025.", source: "PR Newswire", date: "2025-02-12", hot: true }, + ], + }), + wf("lead-nomatch-3", { + "📈 business signals": [ + { description: "Opened a second warehouse; hiring seasonal staff.", source: "company blog", date: "2025-04-20" }, + ], + }), + ], + mission: { + user_intent: + "Find every lead in my Monitor portfolio that acquired a company since 2025, so I can build a campaign.", + success_criteria: [ + "called leadbay_scan_portfolio_signals exactly once to answer the portfolio-wide signal question", + "surfaced QUEST DRAPE, LLC and ACME EVENTS GROUP as the M&A matches with their signal text", + "did NOT loop leadbay_research_lead_by_id per lead to answer the bulk question", + "did NOT claim a lead has or lacks an M&A signal without it appearing in the scan results", + "offered to build a campaign from the matched cohort", + ], + required_calls: ["leadbay_scan_portfolio_signals"], + required_byproducts: [], + forbidden_calls: ["leadbay_report_outreach"], + }, +}; diff --git a/packages/mcp/test/eval/scenarios/scan-portfolio-signals/honest-about-unresearched.scenario.ts b/packages/mcp/test/eval/scenarios/scan-portfolio-signals/honest-about-unresearched.scenario.ts new file mode 100644 index 00000000..a76abdb5 --- /dev/null +++ b/packages/mcp/test/eval/scenarios/scan-portfolio-signals/honest-about-unresearched.scenario.ts @@ -0,0 +1,103 @@ +// Eval scenario — HONESTY / no-fabrication guard for issue #3704. +// +// milstan's diagnosis: "The LLM has bullshitted JM due to profound +// misunderstanding of stale_at values." In JM's run, 30–40% of leads had NO +// cached signal content, yet the agent reported confident verdicts as if it +// had scanned them. This scenario plants a portfolio where some leads are +// genuinely unresearched and asserts the agent is HONEST: it must report those +// leads as not-yet-researched (the scan's not_researched bucket), NOT count +// them as "no M&A signal", and NOT fabricate a verdict for them. +// +// Authored to the README scenario shape (test/eval/README.md). + +const P = (path: string) => `/1.5${path}`; + +const MONITOR_LEADS = [ + { + id: "lead-has-1", + name: "BRIGHT HARBOR LOGISTICS", + score: 80, + location: { city: "Savannah", state: "Georgia", country: "US", full: "Savannah, Georgia, United States" }, + recommended_contact: null, + org_contacts: [], + pushback_status: null, + }, + { + id: "lead-empty-2", + name: "NORTHWIND PARTY CO.", + score: 69, + location: { city: "Boise", state: "Idaho", country: "US", full: "Boise, Idaho, United States" }, + recommended_contact: null, + org_contacts: [], + pushback_status: null, + }, + { + id: "lead-inprogress-3", + name: "CEDAR & OAK RENTALS", + score: 66, + location: { city: "Portland", state: "Oregon", country: "US", full: "Portland, Oregon, United States" }, + recommended_contact: null, + org_contacts: [], + pushback_status: null, + }, +]; + +export const SCENARIO = { + name: "scan-honest-about-unresearched", + prompt: "leadbay_followup_check_in", + tier: "gate", + args: {}, + backendFixtures: [ + { method: "GET", path: P("/monitor/filter"), status: 200, body: { criteria: [] } }, + { + method: "GET", + path: /\/1\.5\/monitor\?/, + status: 200, + body: { items: MONITOR_LEADS, pagination: { page: 0, pages: 1, total: 3 } }, + }, + // lead-has-1: real, researched, NO M&A signal (a true negative). + { + method: "GET", + path: P("/leads/lead-has-1/web_fetch"), + status: 200, + body: { + lead_id: "lead-has-1", + in_progress: false, + fetch_at: "2025-05-01T00:00:00Z", + content: { + "📈 business signals": [ + { description: "Expanded fleet by 12 trucks; no acquisitions reported.", source: "company blog", date: "2025-04-01" }, + ], + }, + }, + }, + // lead-empty-2: never researched — content null. MUST land in not_researched. + { + method: "GET", + path: P("/leads/lead-empty-2/web_fetch"), + status: 200, + body: { lead_id: "lead-empty-2", in_progress: false, fetch_at: null, content: null }, + }, + // lead-inprogress-3: still fetching — MUST land in not_researched too. + { + method: "GET", + path: P("/leads/lead-inprogress-3/web_fetch"), + status: 200, + body: { lead_id: "lead-inprogress-3", in_progress: true, fetch_at: null, content: {} }, + }, + ], + mission: { + user_intent: + "Which of my Monitor leads have an M&A signal? I want to build a campaign from them.", + success_criteria: [ + "called leadbay_scan_portfolio_signals to answer the portfolio-wide question", + "reported that NORTHWIND PARTY CO. and CEDAR & OAK RENTALS are not yet researched (no cached signals), distinct from 'no M&A signal'", + "did NOT claim NORTHWIND PARTY CO. or CEDAR & OAK RENTALS lack an M&A signal — they were never scanned", + "did NOT fabricate any signal, acquisition, or verdict that was not in the scan results", + "offered to qualify the unresearched leads and re-scan, OR to build a campaign from confirmed matches", + ], + required_calls: ["leadbay_scan_portfolio_signals"], + required_byproducts: [], + forbidden_calls: ["leadbay_report_outreach"], + }, +}; diff --git a/packages/mcp/test/output-schema-conformance.test.ts b/packages/mcp/test/output-schema-conformance.test.ts index 448064de..90d29ed7 100644 --- a/packages/mcp/test/output-schema-conformance.test.ts +++ b/packages/mcp/test/output-schema-conformance.test.ts @@ -404,6 +404,35 @@ const CASES: ConformanceCase[] = [ ]); }, }, + { + toolName: "leadbay_scan_portfolio_signals", + arguments: { query: "acquired, M&A", leadIds: ["lead-1"] }, + setupMocks: () => { + mockHttp([ + // Read-only fan-out: a single cached web_fetch read, no POST. + { + method: "GET", + path: "/1.5/leads/lead-1/web_fetch", + status: 200, + body: { + lead_id: "lead-1", + in_progress: false, + fetch_at: "2025-06-01T00:00:00Z", + content: { + "📈 business signals": [ + { + description: "Acme acquired BetaCorp", + source: "techcrunch.com", + date: "2025-03-01", + hot: true, + }, + ], + }, + }, + }, + ]); + }, + }, { toolName: "leadbay_bulk_qualify_leads", arguments: { leadIds: ["lead-1"] }, diff --git a/packages/promptforge/prompts/leadbay_followup_check_in.md.tmpl b/packages/promptforge/prompts/leadbay_followup_check_in.md.tmpl index ad0ef0c3..384ad144 100644 --- a/packages/promptforge/prompts/leadbay_followup_check_in.md.tmpl +++ b/packages/promptforge/prompts/leadbay_followup_check_in.md.tmpl @@ -73,6 +73,21 @@ Unlike `leadbay_daily_check_in` which deep-dives on every promising lead in Phas When the user picks a row, call `leadbay_research_lead_by_id` on that single lead (or `leadbay_research_lead_by_name_fuzzy` if they only have the name) and offer to `leadbay_prepare_outreach` once they say "let's reach out". +# PHASE 3b — BULK SIGNAL SCAN (when the user asks "which of these have signal X") + +If the user wants to filter the whole portfolio by a web-research signal — "which of my leads acquired a company since 2025", "find everyone with a funding signal", "who changed CEO" — do NOT loop `leadbay_research_lead_by_id` per row, and do NOT guess from freshness fields. Call `leadbay_scan_portfolio_signals({query, since?})` once: it bulk-reads the cached signals across the portfolio and returns only the matches, campaign-ready. Offer to build a campaign from the matches. + +**Close the gap — don't just report it.** The scan returns `not_researched[]` (leads with no cached Leadbay signal) and may surface matches whose cached signal is thin or undated. These are the leads where the answer is genuinely unknown, not absent. Rather than stopping at "K aren't researched yet", offer to fill the gap and refine: + +1. **Name the gap precisely** — "N of your M leads matched; K have no cached signal and J more have only a thin/undated mention, so I can't yet confirm signal X for those." +2. **Run a targeted live pass** — if you have web-search tools, research the specific `not_researched` / thin-signal leads for the exact signal the user asked about (company name + the query terms, e.g. " acquisition 2025"). Do this only for the gap leads, not the whole portfolio — the cached matches are already answered. +3. **Fold the findings back in, clearly labelled** — present live results as **agent-sourced (not Leadbay-verified)**, in a section separate from the cached `matched` cohort, and cite the source URL/date you found. Never silently merge a web-found signal into the campaign-ready cohort as if Leadbay had verified it. +4. **Offer the durable path too** — for gap leads worth persisting, offer `leadbay_bulk_qualify_leads` so Leadbay runs its own `web_fetch` and the signal lands in the portfolio's cached `signals[]` on the next scan. + +This keeps the honesty guarantee intact (Leadbay's cached `signals[]` stay the source of truth) while still answering the user's question for the leads Leadbay hasn't researched yet. + +{{include:gates/signal-honesty}} + # CROSS-MODE PIVOT Below the table, offer the cross-mode pivot in one short line so the user can redirect if you guessed wrong on entry-point routing: "Want to see NEW leads from your wishlist instead?" — that routes back to `leadbay_daily_check_in` (Discovery via `leadbay_pull_leads`). diff --git a/packages/promptforge/snippets/gates/signal-honesty.md b/packages/promptforge/snippets/gates/signal-honesty.md new file mode 100644 index 00000000..cb9db93b --- /dev/null +++ b/packages/promptforge/snippets/gates/signal-honesty.md @@ -0,0 +1,8 @@ +**SIGNAL HONESTY — never infer signals from freshness.** `stale_at`, +`web_fetch_in_progress`, `fetch_at` are freshness markers, not signal +indicators — signal presence is read ONLY from the actual `signals[]` / +`web_fetch.content` entries. For "which of my leads have signal X" across a +portfolio, call **`leadbay_scan_portfolio_signals`** (bulk-reads cached +signals); don't loop `leadbay_research_lead_by_id` per lead or guess from +freshness. A lead with no cached content is `not_researched`, not "no match"; +never report a signal verdict for a lead you never read. diff --git a/packages/promptforge/snippets/next-steps/scan-portfolio-signals.md b/packages/promptforge/snippets/next-steps/scan-portfolio-signals.md new file mode 100644 index 00000000..3626ad9e --- /dev/null +++ b/packages/promptforge/snippets/next-steps/scan-portfolio-signals.md @@ -0,0 +1,19 @@ +## NEXT STEPS — after the signal scan + +{{include:next-steps/ask-user-input-routing}} + +The scan exists to BUILD A COHORT, not just to list. The default next move is +almost always "turn the matched leads into a campaign." + +| Observation | Suggest | Calls | +|---------------------------------------------------|--------------------------------------------------------------|----------------------------------------------------------------------------------------| +| `matched` non-empty (top of menu) | "Build a campaign from the N matched leads" | leadbay_create_campaign / leadbay_add_leads_to_campaign(matched lead_ids) | +| `not_researched` non-empty | "K leads aren't researched yet — qualify them, then re-scan" | leadbay_bulk_qualify_leads(not_researched lead_ids) → re-run leadbay_scan_portfolio_signals | +| Zero matches but leads were researched | "Widen the query (synonyms) or relax `since`" | leadbay_scan_portfolio_signals(query: "", since: omit-or-earlier) | +| `truncated_at` set | "Scan only covered N — narrow scope or raise the cap" | leadbay_scan_portfolio_signals({city / set_filter}) or raise `max_leads` | +| One standout matched lead | "Open that lead's full brief" | leadbay_research_lead_by_id(leadId) | +| `quota_exceeded` | "Wait for reset OR top up to finish the scan" | leadbay_create_topup_link | + +NEVER report leads in `not_researched` as if they had no matching signal — they +were never read. Distinguish "no signal X found" (researched, no match) from +"not yet researched" (no data to search) every time. diff --git a/packages/promptforge/snippets/rendering/scan-portfolio-signals.md b/packages/promptforge/snippets/rendering/scan-portfolio-signals.md new file mode 100644 index 00000000..96795393 --- /dev/null +++ b/packages/promptforge/snippets/rendering/scan-portfolio-signals.md @@ -0,0 +1,46 @@ +## RENDERING — bulk signal-scan results + +The output is a cohort, grouped by lead. Lead with the matches, end with an +honesty footer — never hide what wasn't scanned. + +### Matched leads + +Open with a one-line headline: `**N leads match ""** (M scanned).` + +Then one block per `matched[]` lead, ordered with `hot` matches first. Emit +each as a host-parseable per-lead block so the chat host's place-card +auto-detector can render it (per the repo "feed the address auto-detector" +convention): + +``` +### · + + +- ** ** — <🔥 if hot> ([source](), ) +``` + +- **Bold** the description of `hot: true` entries; leave cold entries plain. +- Render `source` as a markdown link `([source](url), date)`; omit the date + when null, omit the link when `source` is empty. +- Cap to the 3 strongest signals per lead (hot first, then by date desc); if a + lead has more, end its block with `_+K more signals_`. +- When `name` is null (the scan was scoped by `leadIds` and the read failed to + carry firmographics), fall back to `### Lead ` — but prefer to enrich + the name via the matched lead's own data when available. + +### Honesty footer (ALWAYS print) + +A single italic line summarising coverage: + +`_Scanned N · matched M · K had no cached signals (not yet researched)._` + +- When `not_researched` is non-empty, this is load-bearing: state plainly that + those K leads were NOT searched and were NOT counted as "no match". Offer to + qualify them and re-scan (see NEXT STEPS). +- When `truncated_at` is set, add: `_Coverage partial — only the first + leads were scanned; narrow the scope or raise max_leads._` +- When `quota_exceeded` is true, add the wait-or-top-up offer. + +**Hide:** raw `lead_id` in prose (use it only for the campaign call), `_meta`, +empty arrays, any freshness field. NEVER present `not_researched` leads as +"no signal found". diff --git a/packages/promptforge/tool-descriptions/composite/pull-followups.md.tmpl b/packages/promptforge/tool-descriptions/composite/pull-followups.md.tmpl index 2a5551ce..5e114b20 100644 --- a/packages/promptforge/tool-descriptions/composite/pull-followups.md.tmpl +++ b/packages/promptforge/tool-descriptions/composite/pull-followups.md.tmpl @@ -51,9 +51,9 @@ annotations: --- Pull KNOWN leads from the user's Monitor view — the re-engagement entry point. Use when the user asks "what should I follow up on", "leads I haven't contacted", "leads in [city]", "before my trip", or any phrasing implying pre-existing pipeline context. For NEW leads from Discover, use `leadbay_pull_leads`. -Backend: wraps `GET /1.5/monitor?personal=&liked=&filtered=&count=&page=` plus, when `set_filter` is supplied, a preceding `POST /1.5/monitor/filter` to persist the filter server-side. The Monitor filter is a single `FilterItem` per user — refreshing the page restores it. +Backend: wraps `GET /1.5/monitor?personal=&liked=&filtered=&count=&page=` plus, when `set_filter` is supplied, a preceding `POST /1.5/monitor/filter`. The Monitor filter is a single `FilterItem` per user — refreshing restores it. -**Filter mechanism — store-then-apply.** Pass `set_filter: { criteria: FilterCriterion[] }` to overwrite the server-stored filter, then the composite re-fetches with `filtered:true`. `FilterCriterion` is the backend's `anyOf` over 10 typed criteria: `size`, `keywords`, `sector_ids`, `location_ids`, `custom_field`, `custom_field_comparison`, `yc`, `liked`, `last_action` (filters by MonitorActionType enum), `last_action_date` (with `last_days` for "last N days"). +**Filter mechanism — store-then-apply.** Pass `set_filter: { criteria: FilterCriterion[] }` to overwrite the server-stored filter, then the composite re-fetches with `filtered:true`. `FilterCriterion` is the backend's `anyOf` over 10 typed criteria: `size`, `keywords`, `sector_ids`, `location_ids`, `custom_field`(`_comparison`), `yc`, `liked`, `last_action` (MonitorActionType enum), `last_action_date` (with `last_days`). Practical mapping from user phrasing to criterion: @@ -66,11 +66,11 @@ Practical mapping from user phrasing to criterion: | "leads 50–200 employees" | `{type: "size", sizes: [{min: 50, max: 200}]}` | | "Y Combinator companies" | `{type: "yc"}` | -Geo filtering needs `admin_area_id` resolution — backend rejects free-text in `location_ids`. Pass `city: ""` and the composite calls `/geo/search` internally, picks the best match, merges its id into `set_filter`. Ambiguous matches return `status: "ambiguous_locations"` + `location_ambiguities[]` — pick an id and re-call with `city_id`. For multi-city cases, call `leadbay_list_locations` then pass `set_filter.criteria` with `{type: "location_ids", is_excluded: false, locations: [...]}`. +Geo filtering needs `admin_area_id` resolution — backend rejects free-text in `location_ids`. Pass `city: ""` and the composite calls `/geo/search` internally, picks the best match, merges its id into `set_filter`. Ambiguous matches return `status: "ambiguous_locations"` + `location_ambiguities[]` — pick an id and re-call with `city_id`. -**Place names go through `city`, NEVER `keywords`.** This includes any geographic token the user names — cities (`"Berlin"`, `"NYC"`), states / provinces / regions (`"Texas"`, `"California"`, `"Bavaria"`), countries (`"France"`, `"United States"`), neighborhoods (`"Brooklyn"`, `"SoHo"`). The `/geo/search` resolver handles all admin levels — level 4 (state) and level 2 (country) resolve just as well as level 5 (city). If you put `"Texas"` in `keywords` you get a TEXT-MATCH against company descriptions (≈0 hits) instead of a real state filter. If a place name resolves ambiguously, surface the choices to the user — do NOT silently fall back to keyword search or to the unfiltered Monitor view. If `keywords: ["Texas"]` returned empty, the next call is `city: "Texas"`, not `keywords: []`. +**Place names go through `city`, NEVER `keywords`.** Any geographic token the user names — cities (`"Berlin"`), states/regions (`"Texas"`, `"Bavaria"`), countries (`"France"`), neighborhoods (`"Brooklyn"`) — resolves via `/geo/search` (all admin levels). A place name in `keywords` becomes a TEXT-MATCH against company descriptions (≈0 hits), not a real filter. If a place resolves ambiguously, surface the choices — never silently fall back to keyword search or the unfiltered view. -**Pushback exclusion.** Leads with active pushback (`pushback_status` set and `pushback_until > today`) are excluded from the response. The composite enforces this client-side; `total_excluded_by_pushback` in the output reports how many rows were dropped. +**Pushback exclusion.** Leads with active pushback (`pushback_status` set, `pushback_until > today`) are excluded client-side; `total_excluded_by_pushback` reports how many rows were dropped. {{include:headers/tool-when-to-use}} re-engaging pipeline ("what should I follow up on", "stale leads"), filtering monitored leads by city / sector / recency / action type / liked. The canonical orchestrator is the `leadbay_followup_check_in` prompt. @@ -78,6 +78,8 @@ Geo filtering needs `admin_area_id` resolution — backend rejects free-text in **Anti-confusion guardrail.** Iterating `pull_leads` pages looking for `prospecting_actions_count > 0` or `notes_count > 0` rows is the wrong entry point — the two read different tables. Leads with follow-up history live in `pull_followups`. +{{include:gates/signal-honesty}} + --- {{include:rendering/pull-followups-table}} diff --git a/packages/promptforge/tool-descriptions/composite/research-lead-by-id.md.tmpl b/packages/promptforge/tool-descriptions/composite/research-lead-by-id.md.tmpl index 37657492..7787036d 100644 --- a/packages/promptforge/tool-descriptions/composite/research-lead-by-id.md.tmpl +++ b/packages/promptforge/tool-descriptions/composite/research-lead-by-id.md.tmpl @@ -35,42 +35,38 @@ annotations: openWorldHint: true --- Tell me everything decision-relevant about a single lead, identified by its -Leadbay UUID. Bundles the lens-scoped lead profile, the AI qualification -answers (the agent's knowledge-base food), the structured web-research signals -(with hot flags + sources), the two-tier contact set (`enriched` + `org`), the -unified `recent_activities` timeline, the engagement counts, and a -`_meta.has_reachable_contact` hint that drives NEXT STEPS. Order is -deliberate: qualification first, then signals, then firmographics, then -contacts, then recent activity. +Leadbay UUID. Bundles the lens-scoped profile, AI qualification answers, +structured web-research signals (hot flags + sources), the two-tier contact set +(`enriched` + `org`), the unified `recent_activities` timeline, engagement +counts, and a `_meta.has_reachable_contact` hint that drives NEXT STEPS. Order +is deliberate: qualification, signals, firmographics, contacts, recent activity. -Scoring has two layers: the basic `score` (firmographic, always present, -already decent) and the AI qualification layer (`ai_agent_lead_score` + -per-question answers + web_fetch signals). The AI layer is pre-populated for -roughly the top 10 of each daily batch, and on-demand (via -leadbay_bulk_qualify_leads) for anything below that. Combine both layers when -judging a lead. +Scoring has two layers: the basic `score` (firmographic, always present) and +the AI qualification layer (`ai_agent_lead_score` + per-question answers + +web_fetch signals). The AI layer is pre-populated for roughly the top 10 of +each daily batch, and on-demand (via leadbay_bulk_qualify_leads) below that. +Combine both when judging a lead. -The companion tool **leadbay_research_lead_by_name_fuzzy** wraps this one for -the case where the user names a company in prose without a UUID — it -fuzzy-resolves the name against the active lens's wishlist, then delegates -here. Both return the same shape; the fuzzy wrapper just adds -`_meta.resolved_from` and `_meta.match_candidates` so you can offer -disambiguation. +The companion **leadbay_research_lead_by_name_fuzzy** wraps this one when the +user names a company without a UUID: it fuzzy-resolves against the active +lens's wishlist, then delegates here. Same shape, plus `_meta.resolved_from` / +`_meta.match_candidates`. {{include:headers/tool-when-to-use}} when picking up a single lead from leadbay_pull_leads (or any list that exposed a leadId) to decide whether to act on it. {{include:headers/tool-when-not-to-use}} across many leads at once — that's -leadbay_pull_leads' job. (This composite supersedes the lower-level -leadbay_get_lead_profile in agent flow; the granular tool stays available for -fine-grained access.) +leadbay_pull_leads' job (portfolio-wide signal questions go to +leadbay_scan_portfolio_signals; see below). This composite supersedes the +lower-level leadbay_get_lead_profile. -**Concurrency note**: this is a composite that reads many sub-resources per -call. Call it **sequentially** or in small batches (≤3 parallel) when -researching multiple leads. Firing 10+ in parallel can saturate the transport -and produce misleading `"Tool permission stream closed"` errors that look like -permission failures but are really backpressure. On a transient +{{include:gates/signal-honesty}} + +**Concurrency note**: this composite reads many sub-resources per call. Call +it **sequentially or in small batches (≤3 parallel)**. Firing 10+ in parallel +saturates the transport and produces misleading `"Tool permission stream +closed"` errors — that's backpressure, not a permission failure. On a transient stream/timeout failure, retry the same lead once before moving on. --- diff --git a/packages/promptforge/tool-descriptions/composite/scan-portfolio-signals.md.tmpl b/packages/promptforge/tool-descriptions/composite/scan-portfolio-signals.md.tmpl new file mode 100644 index 00000000..8ac5a51e --- /dev/null +++ b/packages/promptforge/tool-descriptions/composite/scan-portfolio-signals.md.tmpl @@ -0,0 +1,104 @@ +--- +name: leadbay_scan_portfolio_signals +kind: tool-description +short_description: | + Scan a KNOWN portfolio (Monitor view, or an explicit lead-id list) for a + specific web-research signal in bulk — e.g. "which of my leads acquired a + company since 2025". Reads CACHED signals across many leads in one call and + returns only the matches, campaign-ready. Use instead of looping + leadbay_research_lead_by_id per lead. Don't use it to research one named + company (that's leadbay_research_lead_by_name_fuzzy) or to qualify + unresearched leads (that's leadbay_bulk_qualify_leads). +routing: + triggers: + - "which of my leads " + - "find leads that " + - "scan my portfolio for " + - "identify all the ones that since " + - "who in Monitor has a signal" + - "build a campaign from leads with " + anti_triggers: + - phrase: "research one named company" + route_to: leadbay_research_lead_by_name_fuzzy + - phrase: "everything about lead " + route_to: leadbay_research_lead_by_id + - phrase: "qualify my next N leads (they aren't researched yet)" + route_to: leadbay_bulk_qualify_leads + - phrase: "just list my follow-ups" + route_to: leadbay_pull_followups + prefer_when: "user wants to FILTER a known portfolio by a web-research signal in bulk — pass `query`, optionally `since`, `city`/`set_filter`, or `leadIds`" + examples: + positive: + - "Which of my leads acquired a company since 2025?" + - "Scan my Lyon portfolio for funding signals." + - "Find everyone in Monitor who changed CEO and build a campaign." + negative: + - "Look up Acme Corp for me." + - "Show me my follow-ups." + - "Qualify my next 10 leads." +rendering_hint: | + Cohort grouped by lead: one block per matched lead (name · location + + its matched signal entries, hot first, source-linked). Open with + "N match (M scanned)"; ALWAYS close with an honesty footer — + "scanned N · matched M · K not yet researched". Never present + not_researched leads as "no signal". Full layout below. +next_steps: scan-portfolio-signals +annotations: + readOnlyHint: true + destructiveHint: false + idempotentHint: true + openWorldHint: true +--- +Scan a known portfolio for a specific web-research signal in one call. This is +the bulk, read-only answer to "which of my leads have signal X" — the question +that otherwise forces a per-lead `leadbay_research_lead_by_id` loop (one full +profile call per lead, slow and quota-heavy). + +**Reads CACHED signals only — does not trigger new research.** For each lead in +scope it reads `GET /leads/{id}/web_fetch` (the already-computed web-research +signals) and filters the entries against `query`. It issues NO web_fetch POST, +so it does not consume AI qualification credits and does not re-crawl. Leads +that have no cached content (never qualified, or still in progress) are +reported in `not_researched` — they are **NOT** silently treated as "no +match". Qualify them with `leadbay_bulk_qualify_leads`, then re-scan. + +**Scope.** Pass `leadIds` for an explicit cohort, or omit it to scan the +Monitor portfolio. Narrow the Monitor scope with `city` / `set_filter` exactly +as `leadbay_pull_followups` does (store-then-apply server-side filter). The +scan is bounded by `max_leads` (default 200, hard cap 300); when the portfolio +is larger, `truncated_at` is set and coverage is partial — say so. + +**Query.** `query` is matched case- and accent-insensitively against each +signal entry's description, source, and section label. Comma- or +space-separated terms are OR'd ("M&A, acquisition, racheté" matches any). Use +`since` (ISO date) to keep only entries dated on/after it — entries with no +date are kept (a missing date is not evidence the event is old). + +**Result is campaign-ready.** `matched[]` carries `lead_id`, `name`, +`location`, and the matching `matched_signals[]` (section + hot + source + +date + description). Feed the matched `lead_id`s straight into +`leadbay_add_leads_to_campaign` / `leadbay_create_campaign`. + +On a 429 mid-scan, partial `matched` is returned with `quota_exceeded: true` — +offer the user wait-for-reset OR a top-up link (both unblock; a top-up clears +the throttle immediately). + +{{include:gates/signal-honesty}} + +{{include:headers/tool-when-to-use}} when the user wants to filter a known +portfolio by a web-research signal across many leads at once — discovering a +cohort to act on, not inspecting a single lead. + +{{include:headers/tool-when-not-to-use}} for a single named company +(leadbay_research_lead_by_name_fuzzy) or one lead by UUID +(leadbay_research_lead_by_id); to qualify leads that have no signals yet +(leadbay_bulk_qualify_leads); or to just list follow-ups with no signal filter +(leadbay_pull_followups). + +--- + +{{include:rendering/scan-portfolio-signals}} + +--- + +{{include:next-steps/scan-portfolio-signals}}