feat(ai-chat): route Auto model by task signals#599
Conversation
Auto currently resolves every turn to one hardcoded gateway model. Add a pure, deterministic router: task signals in, standard-tier picker model out, driven by a data table of thresholds and rules. Targets that go missing or leave the standard billing tier degrade to the existing default model instead of failing the turn or reaching premium pricing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Derive the router's task signals (message and text volume, image attachments, prior tool calls, code markers, workspace references) from the assembled turn context and request body. Extraction is best-effort: missing or malformed fields become zero values so a bad payload can never fail the turn. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When the resolved picker id is auto, beforeTurn now routes the turn from extracted task signals and uses the routed id for the language model, gateway provider options, access check, and usage context — so metering follows the model that actually ran. Continuation turns reuse the id routed at run start, and explicit picker choices pass through untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Auto turns now report both the picker selection and the routed model: PostHog ai_turn_started gains requested_model_id and routing_reason, TCC run metadata carries the same pair, and the inspector records them on turn.started, parses them into the run view, and shows requested vs routed in the run summary badge. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The picker described Auto as picking a good fit while it always called one hardcoded model. Now that turns are actually routed, say so plainly and note in the catalog comment that the auto gateway slug is the router's default/fallback and the base model for non-chat paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughIntroduces a rule-based auto model router for workspace AI chat that selects a gateway model based on request signals (attachments, context length, tool usage). Wires routing into ChangesAuto Model Routing
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant AIThread
participant ModelRouter as model-router
participant TelemetryRecorders as Telemetry/PostHog/TCC recorders
participant Inspector as AI Inspector UI
AIThread->>AIThread: beforeTurn resolves requestedModelId
AIThread->>ModelRouter: extractWorkspaceAiRoutingSignals(ctx)
ModelRouter-->>AIThread: signals
AIThread->>ModelRouter: routeWorkspaceAiAutoModel(signals)
ModelRouter-->>AIThread: { modelId, reason }
AIThread->>TelemetryRecorders: recordTurnStarted(modelId, requestedModelId, routingReason)
TelemetryRecorders->>TelemetryRecorders: emit requested_model_id, routing_reason
AIThread->>Inspector: turn.started event payload includes requestedModelId, routingReason
Inspector->>Inspector: render requestedModelId → modelId badge
Suggested labels: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThis PR adds task-aware routing for the Auto AI chat model. The main changes are:
Confidence Score: 5/5Safe to merge with low risk. The changes are localized to Auto model routing and telemetry, the router is pure and defensive, and standard-tier fallback behavior is covered by tests. No files require special attention.
 T-Rex LogsWhat T-Rex did
Important Files Changed
Reviews (1): Last reviewed commit: "fix(ai-chat): align Auto picker copy wit..." | Re-trigger Greptile |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (1)
src/features/workspaces/ai/model-router.ts (1)
56-63: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick winExclude
autofrom rule targets.
WorkspaceAiChatModelIdincludes the user-facing"auto"id, andgetStandardTierRouteTargetcurrently accepts it because it is standard-tier. A future rule tuned totargetModelId: "auto"would be reported as a successful route with a non-fallback reason instead of a concrete backend choice.Proposed tightening
export interface WorkspaceAiAutoRoutingRule { matches: ( signals: WorkspaceAiRoutingSignals, thresholds: WorkspaceAiAutoRoutingThresholds, ) => boolean; reason: Exclude<WorkspaceAiAutoRouteReason, "high-reasoning-default" | "fallback-unavailable">; - targetModelId: WorkspaceAiChatModelId; + targetModelId: Exclude<WorkspaceAiChatModelId, "auto">; }function getStandardTierRouteTarget(modelId: WorkspaceAiChatModelId) { const model = getWorkspaceAiChatModelById(modelId); // getWorkspaceAiChatModelById falls back to the default entry for unknown // ids; treat that as "unavailable" rather than silently routing there. - if (model.id !== modelId || model.billingTier !== "standard") { + if (modelId === "auto" || model.id !== modelId || model.billingTier !== "standard") { return undefined; }Also applies to: 139-148
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/features/workspaces/ai/model-router.ts` around lines 56 - 63, The routing rule target type currently allows the user-facing "auto" model id, which can be returned as if it were a concrete backend choice. Tighten the type used by WorkspaceAiAutoRoutingRule.targetModelId and the related getStandardTierRouteTarget path so rules can only point to actual backend models, excluding "auto" while keeping standard-tier validation intact. Use the existing symbols WorkspaceAiAutoRoutingRule, WorkspaceAiChatModelId, and getStandardTierRouteTarget to update the rule target typing and any affected checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/features/workspaces/ai/ai-thread.ts`:
- Around line 318-332: The explicit-picker continuation path in
_resolveAutoModelRoute is returning undefined, which lets beforeTurn fall back
to the Auto default instead of preserving the original model. Update
_resolveAutoModelRoute so that when ctx.continuation is true and
this.activeUsageContext.requestedModelId is not "auto", it routes back to that
original explicit modelId with an appropriate reason. Keep the existing
auto-originated continuation handling intact, and make sure beforeTurn still
uses route?.modelId to prevent mid-flight model switches.
- Around line 333-345: Persist or rehydrate the routed context in ai-thread.ts
so resumed continuations keep an activeUsageContext instead of only rerouting
from current signals. Update the logic around the existing activeContext check
and the routeWorkspaceAiAutoModel/extractWorkspaceAiRoutingSignals path so the
returned model/thread routing data is also stored on the thread state and
available to _trackCompletedMessageUsage. This should ensure hibernation or
recovery restores the same usage context before completed-message billing runs.
In `@src/features/workspaces/ai/model-router.test.ts`:
- Around line 249-265: The test expectation in extractWorkspaceAiRoutingSignals
should be updated to match the new parsed-message counting behavior for
malformed turn data. Adjust the fixture in model-router.test.ts so it no longer
asserts messageCount: 3 when null/invalid messages are ignored, and keep the
rest of the expected routing signals aligned with the output from
buildTurnContext and extractWorkspaceAiRoutingSignals.
In `@src/features/workspaces/ai/model-router.ts`:
- Around line 180-205: The message count is currently using the raw array length
in the model-router message analysis, which can include malformed entries that
were skipped earlier. Update the message-count logic in the workspace AI routing
helper around the loop that uses asRecord, getMessageText, and countParts so it
counts only successfully parsed records rather than messages.length, and return
that parsed count from the same object that includes attachmentCount,
hasCodeSignals, lastUserTextChars, and priorToolCallCount.
---
Nitpick comments:
In `@src/features/workspaces/ai/model-router.ts`:
- Around line 56-63: The routing rule target type currently allows the
user-facing "auto" model id, which can be returned as if it were a concrete
backend choice. Tighten the type used by
WorkspaceAiAutoRoutingRule.targetModelId and the related
getStandardTierRouteTarget path so rules can only point to actual backend
models, excluding "auto" while keeping standard-tier validation intact. Use the
existing symbols WorkspaceAiAutoRoutingRule, WorkspaceAiChatModelId, and
getStandardTierRouteTarget to update the rule target typing and any affected
checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 2dbb9319-1704-4da0-b644-e1176d5a86fc
📒 Files selected for processing (13)
src/features/workspaces/ai/ai-inspector-view-model.test.tssrc/features/workspaces/ai/ai-inspector-view-model.tssrc/features/workspaces/ai/ai-inspector-view-types.tssrc/features/workspaces/ai/ai-thread-inspector-recorder.tssrc/features/workspaces/ai/ai-thread-posthog-recorder.tssrc/features/workspaces/ai/ai-thread-tcc-recorder.tssrc/features/workspaces/ai/ai-thread-telemetry-recorder.tssrc/features/workspaces/ai/ai-thread.tssrc/features/workspaces/ai/model-router.test.tssrc/features/workspaces/ai/model-router.tssrc/features/workspaces/ai/models.tssrc/features/workspaces/components/ai-chat/AiChatInspectorViews.tsxsrc/integrations/posthog/events.ts
| private _resolveAutoModelRoute( | ||
| requestedModelId: WorkspaceAiChatModelId, | ||
| ctx: TurnContext, | ||
| ): { modelId: WorkspaceAiChatModelId; reason: string } | undefined { | ||
| if (requestedModelId !== "auto") { | ||
| return undefined; | ||
| } | ||
|
|
||
| const activeContext = ctx.continuation ? this.activeUsageContext : undefined; | ||
|
|
||
| if (activeContext && activeContext.requestedModelId !== "auto") { | ||
| // The run started from an explicit picker choice; a continuation | ||
| // whose body lost the selection must not be re-routed. | ||
| return undefined; | ||
| } |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Explicit-picker continuation branch still switches models mid-flight.
When a continuation's activeContext.requestedModelId !== "auto" (original turn used an explicit model, e.g. claude-sonnet), this branch returns undefined. Since beforeTurn computes modelId = route?.modelId ?? requestedModelId, and requestedModelId for this continuation call is "auto" (the body lost the explicit selection), the effective model silently becomes Auto's default (moonshotai/kimi-k2.6) instead of the originally selected explicit model. This directly contradicts the function's own header comment that "a run never switches models mid-flight" — continuity is only preserved for the auto-originated branch (lines 334-339), not for the explicit-picker branch.
🐛 Proposed fix to preserve the original explicit model
if (activeContext && activeContext.requestedModelId !== "auto") {
- // The run started from an explicit picker choice; a continuation
- // whose body lost the selection must not be re-routed.
- return undefined;
+ // The run started from an explicit picker choice; a continuation
+ // whose body lost the selection must reuse that model, not silently
+ // fall back to Auto's default.
+ return { modelId: activeContext.modelId, reason: "turn-continuation" };
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| private _resolveAutoModelRoute( | |
| requestedModelId: WorkspaceAiChatModelId, | |
| ctx: TurnContext, | |
| ): { modelId: WorkspaceAiChatModelId; reason: string } | undefined { | |
| if (requestedModelId !== "auto") { | |
| return undefined; | |
| } | |
| const activeContext = ctx.continuation ? this.activeUsageContext : undefined; | |
| if (activeContext && activeContext.requestedModelId !== "auto") { | |
| // The run started from an explicit picker choice; a continuation | |
| // whose body lost the selection must not be re-routed. | |
| return undefined; | |
| } | |
| private _resolveAutoModelRoute( | |
| requestedModelId: WorkspaceAiChatModelId, | |
| ctx: TurnContext, | |
| ): { modelId: WorkspaceAiChatModelId; reason: string } | undefined { | |
| if (requestedModelId !== "auto") { | |
| return undefined; | |
| } | |
| const activeContext = ctx.continuation ? this.activeUsageContext : undefined; | |
| if (activeContext && activeContext.requestedModelId !== "auto") { | |
| // The run started from an explicit picker choice; a continuation | |
| // whose body lost the selection must reuse that model, not silently | |
| // fall back to Auto's default. | |
| return { modelId: activeContext.modelId, reason: "turn-continuation" }; | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/features/workspaces/ai/ai-thread.ts` around lines 318 - 332, The
explicit-picker continuation path in _resolveAutoModelRoute is returning
undefined, which lets beforeTurn fall back to the Auto default instead of
preserving the original model. Update _resolveAutoModelRoute so that when
ctx.continuation is true and this.activeUsageContext.requestedModelId is not
"auto", it routes back to that original explicit modelId with an appropriate
reason. Keep the existing auto-originated continuation handling intact, and make
sure beforeTurn still uses route?.modelId to prevent mid-flight model switches.
|
|
||
| if (activeContext) { | ||
| return { | ||
| modelId: activeContext.modelId, | ||
| reason: activeContext.routingReason ?? "turn-continuation", | ||
| }; | ||
| } | ||
|
|
||
| const routed = routeWorkspaceAiAutoModel(extractWorkspaceAiRoutingSignals(ctx)); | ||
|
|
||
| return { modelId: routed.modelId, reason: routed.reason }; | ||
| } | ||
|
|
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Check if activeUsageContext (or equivalent) is persisted to durable storage anywhere,
# and how "continuation" is defined/triggered relative to DO hibernation.
rg -n "activeUsageContext" -C3 --type=ts
rg -n "continuation" -C3 --type=ts src/features/workspaces/ai/ai-thread.tsRepository: ThinkEx-OSS/thinkex
Length of output: 157
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Map the target file and read the relevant sections.
wc -l src/features/workspaces/ai/ai-thread.ts
sed -n '160,460p' src/features/workspaces/ai/ai-thread.ts
# Search for persistence/restoration and continuation-related paths.
rg -n "activeUsageContext|usageContext|routeWorkspaceAiAutoModel|extractWorkspaceAiRoutingSignals|beforeTurn|_trackCompletedMessageUsage|continuation|durable|storage|hibernate|hibernation|restore|recover" src/features/workspaces/ai -C 3
rg -n "activeUsageContext|usageContext|continuation|hibernate|hibernation|durable|storage|restore|recover" src -C 2Repository: ThinkEx-OSS/thinkex
Length of output: 50377
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Find durable-object / hibernation / state-restoration references near AIThread.
rg -n "hibernate|hibernation|durable object|Durable Object|storage|persist|restore|recover|recovery" src/features/workspaces/ai -C 2
# Show the AIThread class definition and any storage-related fields/methods.
sed -n '1,160p' src/features/workspaces/ai/ai-thread.tsRepository: ThinkEx-OSS/thinkex
Length of output: 13924
🌐 Web query:
Cloudflare Think Durable Object continuation recovery instance fields hibernation activeUsageContext beforeTurn continuation onChatResponse
💡 Result:
In the context of Cloudflare Agents and Durable Objects, the terms you mentioned pertain to the lifecycle, lifecycle hooks, and operational behavior of agentic applications built on the Cloudflare platform [1][2]. ActiveUsageContext and BeforeTurn BeforeTurn is a lifecycle hook in the Cloudflare Agents framework that fires before streamText is executed [3][2]. It receives a TurnContext object containing the assembled system prompt, messages, tools, and model. This hook allows developers to return a TurnConfig object to override parts of the turn (e.g., model, system prompt, tools, maxSteps) [3][2]. Continuation Continuation refers to a specific type of turn in an agent's lifecycle, often used for auto-continuing a conversation after tool results are coalesced [3][2]. In the TurnContext provided to lifecycle hooks like beforeTurn, the continuation field is a boolean indicating whether the current turn is an auto-continuation [2]. Hibernation and Durable Objects Durable Objects support WebSocket Hibernation, which allows objects to be removed from memory to save costs while maintaining active WebSocket connections [4][5]. When an agent relies on Durable Objects for state, hibernation can impact certain behaviors: - Programmatic turns and sub-agent RPCs may be affected, as certain signals (like AbortSignal) do not persist across hibernation boundaries [6]. - Lifecycle hooks and responses: Some responses triggered from within hooks like onChatResponse do not recursively fire onChatResponse again, which is a consideration when designing agent communication flows [7][8]. Recovery Durable recovery refers to mechanisms designed to manage stability and handle interruptions (such as stream-stall watchdogs) [1]. The chatStreamStallTimeoutMs field in TurnConfig is specifically mentioned as a way to override the watchdog for a specific turn, such as when executing a known-slow tool, and it is explicitly noted to refer to Durable recovery documentation [2][9].
Citations:
- 1: https://developers.cloudflare.com/agents/harnesses/think/
- 2: https://developers.cloudflare.com/agents/harnesses/think/lifecycle-hooks/
- 3: cloudflare/agents@8c7caab
- 4: https://developers.cloudflare.com/durable-objects/api/state/
- 5: https://thomasgauvin.com/writing/how-cloudflare-durable-objects-websocket-hibernation-works/
- 6: https://developers.cloudflare.com/agents/harnesses/think/sub-agents/
- 7: https://developers.cloudflare.com/agents/communication-channels/chat/chat-agents/
- 8: https://github.com/cloudflare/cloudflare-docs/blob/production/src/content/docs/agents/api-reference/chat-agents.mdx
- 9: https://developers.cloudflare.com/agents/think/lifecycle-hooks/
🌐 Web query:
Cloudflare Durable Objects instance fields lost on hibernation state.storage persistence docs
💡 Result:
When a Cloudflare Durable Object enters a hibernation state—or is evicted from memory due to inactivity or system events like code deployments—all in-memory instance fields and variables are discarded [1][2][3][4]. This occurs because the object is removed from the active memory of the runtime [1][5]. To maintain state across these events, you must explicitly persist any necessary information to the Durable Object's durable storage using the Storage API [1][2][3]. The Durable Object Storage API provides transactional, strongly consistent, and private storage (including SQL and key-value options) that survives hibernation and eviction [1][6][5][7]. Special handling is required for WebSocket Hibernation: * While general in-memory state is lost, you can use the Hibernation WebSocket API methods serializeAttachment and deserializeAttachment to persist per-connection data [8][9]. * These attachments survive hibernation as long as the WebSocket connection remains healthy [8]. * If you need to store larger amounts of data or information that must persist beyond the lifetime of a specific WebSocket connection, you should use the standard Storage API and store only the reference key in the WebSocket attachment [8]. In summary, you should treat all instance variables as ephemeral and assume they will be re-initialized from persistent storage upon the next request [1][4].
Citations:
- 1: https://developers.cloudflare.com/durable-objects/best-practices/access-durable-objects-storage/
- 2: https://developers.cloudflare.com/durable-objects/concepts/durable-object-lifecycle/
- 3: https://developers.cloudflare.com/durable-objects/reference/in-memory-state/
- 4: https://developers.cloudflare.com/durable-objects/examples/durable-object-in-memory-state/index.md
- 5: https://developers.cloudflare.com/durable-objects/concepts/what-are-durable-objects/
- 6: https://developers.cloudflare.com/durable-objects/api/sqlite-storage-api/index.md
- 7: https://developers.cloudflare.com/durable-objects/best-practices/access-durable-objects-storage/index.md
- 8: https://developers.cloudflare.com/durable-objects/best-practices/websockets/
- 9: https://developers.cloudflare.com/durable-objects/examples/websocket-hibernation-server/index.md
Persist activeUsageContext for resumed continuations. A fresh instance can reroute the turn from current signals, but _trackCompletedMessageUsage will then skip billing because activeUsageContext is still unset. Store or rehydrate the routed model/thread context so hibernation or recovery doesn’t drop usage tracking for completed runs.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/features/workspaces/ai/ai-thread.ts` around lines 333 - 345, Persist or
rehydrate the routed context in ai-thread.ts so resumed continuations keep an
activeUsageContext instead of only rerouting from current signals. Update the
logic around the existing activeContext check and the
routeWorkspaceAiAutoModel/extractWorkspaceAiRoutingSignals path so the returned
model/thread routing data is also stored on the thread state and available to
_trackCompletedMessageUsage. This should ensure hibernation or recovery restores
the same usage context before completed-message billing runs.
| it("returns zero signals for missing or malformed turn data", () => { | ||
| expect( | ||
| extractWorkspaceAiRoutingSignals( | ||
| buildTurnContext({ | ||
| body: { workspaceAiContext: "garbage" }, | ||
| messages: [null, 42, { role: "user", content: { nested: true } }], | ||
| } as unknown as Partial<TurnContext>), | ||
| ), | ||
| ).toEqual({ | ||
| attachmentCount: 0, | ||
| hasCodeSignals: false, | ||
| lastUserTextChars: 0, | ||
| messageCount: 3, | ||
| priorToolCallCount: 0, | ||
| totalTextChars: 0, | ||
| workspaceReferenceCount: 0, | ||
| }); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Update this expectation with parsed-message counting.
If extractWorkspaceAiRoutingSignals stops counting ignored malformed entries, this fixture should no longer expect messageCount: 3; otherwise it locks in the inflated-count behavior.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/features/workspaces/ai/model-router.test.ts` around lines 249 - 265, The
test expectation in extractWorkspaceAiRoutingSignals should be updated to match
the new parsed-message counting behavior for malformed turn data. Adjust the
fixture in model-router.test.ts so it no longer asserts messageCount: 3 when
null/invalid messages are ignored, and keep the rest of the expected routing
signals aligned with the output from buildTurnContext and
extractWorkspaceAiRoutingSignals.
| for (const message of messages) { | ||
| const record = asRecord(message); | ||
|
|
||
| if (!record) { | ||
| continue; | ||
| } | ||
|
|
||
| const text = getMessageText(record.content); | ||
| totalTextChars += text.length; | ||
|
|
||
| if (record.role === "user") { | ||
| attachmentCount += countParts(record.content, ["file", "image"]); | ||
| lastUserText = text; | ||
| } else if (record.role === "assistant") { | ||
| priorToolCallCount += countParts(record.content, ["tool-call"]); | ||
| } | ||
| } | ||
|
|
||
| return { | ||
| attachmentCount, | ||
| hasCodeSignals: WORKSPACE_AI_CODE_SIGNAL_PATTERNS.some((pattern) => | ||
| pattern.test(lastUserText), | ||
| ), | ||
| lastUserTextChars: lastUserText.length, | ||
| messageCount: messages.length, | ||
| priorToolCallCount, |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Count parsed messages, not raw array entries.
messageCount: messages.length lets malformed entries like null/numbers inflate routing signals and can trigger the long-context rule even though those entries were ignored during extraction.
Proposed fix
let priorToolCallCount = 0;
let totalTextChars = 0;
let lastUserText = "";
+ let messageCount = 0;
for (const message of messages) {
const record = asRecord(message);
if (!record) {
continue;
}
+ messageCount += 1;
+
const text = getMessageText(record.content);
totalTextChars += text.length;
@@
lastUserTextChars: lastUserText.length,
- messageCount: messages.length,
+ messageCount,
priorToolCallCount,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| for (const message of messages) { | |
| const record = asRecord(message); | |
| if (!record) { | |
| continue; | |
| } | |
| const text = getMessageText(record.content); | |
| totalTextChars += text.length; | |
| if (record.role === "user") { | |
| attachmentCount += countParts(record.content, ["file", "image"]); | |
| lastUserText = text; | |
| } else if (record.role === "assistant") { | |
| priorToolCallCount += countParts(record.content, ["tool-call"]); | |
| } | |
| } | |
| return { | |
| attachmentCount, | |
| hasCodeSignals: WORKSPACE_AI_CODE_SIGNAL_PATTERNS.some((pattern) => | |
| pattern.test(lastUserText), | |
| ), | |
| lastUserTextChars: lastUserText.length, | |
| messageCount: messages.length, | |
| priorToolCallCount, | |
| let priorToolCallCount = 0; | |
| let totalTextChars = 0; | |
| let lastUserText = ""; | |
| let messageCount = 0; | |
| for (const message of messages) { | |
| const record = asRecord(message); | |
| if (!record) { | |
| continue; | |
| } | |
| messageCount += 1; | |
| const text = getMessageText(record.content); | |
| totalTextChars += text.length; | |
| if (record.role === "user") { | |
| attachmentCount += countParts(record.content, ["file", "image"]); | |
| lastUserText = text; | |
| } else if (record.role === "assistant") { | |
| priorToolCallCount += countParts(record.content, ["tool-call"]); | |
| } | |
| } | |
| return { | |
| attachmentCount, | |
| hasCodeSignals: WORKSPACE_AI_CODE_SIGNAL_PATTERNS.some((pattern) => | |
| pattern.test(lastUserText), | |
| ), | |
| lastUserTextChars: lastUserText.length, | |
| messageCount, | |
| priorToolCallCount, |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/features/workspaces/ai/model-router.ts` around lines 180 - 205, The
message count is currently using the raw array length in the model-router
message analysis, which can include malformed entries that were skipped earlier.
Update the message-count logic in the workspace AI routing helper around the
loop that uses asRecord, getMessageText, and countParts so it counts only
successfully parsed records rather than messages.length, and return that parsed
count from the same object that includes attachmentCount, hasCodeSignals,
lastUserTextChars, and priorToolCallCount.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 13 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/features/workspaces/ai/model-router.ts">
<violation number="1" location="src/features/workspaces/ai/model-router.ts:204">
P2: Malformed or non-record messages are skipped when computing text, attachments, and tool-call signals, but `messageCount` is set to the raw `messages.length` regardless. This means invalid entries still count toward routing thresholds—for example, an array of 40 malformed messages yields `messageCount: 40` and incorrectly triggers the `long-context` rule, routing Auto to Gemini instead of the default route. It can also incorrectly suppress the `simple-chat` rule. To match the defensive contract that malformed input produces zero-value signals, only valid messages should be counted.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| pattern.test(lastUserText), | ||
| ), | ||
| lastUserTextChars: lastUserText.length, | ||
| messageCount: messages.length, |
There was a problem hiding this comment.
P2: Malformed or non-record messages are skipped when computing text, attachments, and tool-call signals, but messageCount is set to the raw messages.length regardless. This means invalid entries still count toward routing thresholds—for example, an array of 40 malformed messages yields messageCount: 40 and incorrectly triggers the long-context rule, routing Auto to Gemini instead of the default route. It can also incorrectly suppress the simple-chat rule. To match the defensive contract that malformed input produces zero-value signals, only valid messages should be counted.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/features/workspaces/ai/model-router.ts, line 204:
<comment>Malformed or non-record messages are skipped when computing text, attachments, and tool-call signals, but `messageCount` is set to the raw `messages.length` regardless. This means invalid entries still count toward routing thresholds—for example, an array of 40 malformed messages yields `messageCount: 40` and incorrectly triggers the `long-context` rule, routing Auto to Gemini instead of the default route. It can also incorrectly suppress the `simple-chat` rule. To match the defensive contract that malformed input produces zero-value signals, only valid messages should be counted.</comment>
<file context>
@@ -0,0 +1,263 @@
+ pattern.test(lastUserText),
+ ),
+ lastUserTextChars: lastUserText.length,
+ messageCount: messages.length,
+ priorToolCallCount,
+ totalTextChars,
</file context>
2 participants
Closes #570.
Root cause
Autois the default picker choice but resolves to one hardcoded backend (gatewayModel: "moonshotai/kimi-k2.6"inmodels.ts), while the picker copy claims "Picks a good fit for you". There was no task-aware selection step anywhere between the picker and the gateway.Approach and routing policy
Routing resolves at turn time only, in
AIThread.beforeTurn, through a new pure router (src/features/workspaces/ai/model-router.ts). The user-facing id stays"auto"everywhere (picker state, persisted UI store, request body). When the resolved id isauto,beforeTurnextracts task signals from the assembled turn context and routes; the routed picker id then flows into the existing helpers for the language model, gateway provider options, the access check, usage metering, and telemetry. Non-auto picks pass through untouched.Policy is a data table (thresholds + rules), first match wins; capability rules precede cost rules:
attachment-heavygemini(Gemini 3 Flash)long-contextgeminitool-heavychatgpt-minisimple-chatclaude-haikuhigh-reasoning-defaultauto(Kimi K2.6)fallback-unavailableauto(Kimi K2.6)Billing: routing is deliberately standard-tier-only — a rule can never land on a premium model; a re-tiered or removed target degrades to the default at route time (
fallback-unavailable, covered by tests). Usage is metered on the routed model id viaactiveUsageContext, which now carries bothrequestedModelIdand the routedmodelId; under standard-only routing this maps to the same Autumn feature id as before and stays correct by construction if policy ever changes. Premium-tier routing is a product decision deliberately left to the maintainer. The (currently disabled) access check receives the routed id, with a comment at the call site that enforcement must gate on the routed tier.Telemetry/inspector:
recordTurnStartednow takesmodelId(routed) plusrequestedModelId/routingReason. PostHogai_turn_startedgainsrequested_model_idandrouting_reason; TCC run metadata carries the same pair; the inspector records them onturn.started, parses them into the run view, and the run summary badge showsauto → claude-haikuplus the reason. Because the routed id is a picker id, gateway tags, per-model reasoning options, and gateway-model derivation in every recorder stay correct with no extra plumbing.Determinism/latency: the router and signal extraction are pure and synchronous — zero model calls, zero IO. Extraction is defensive: malformed messages or body fields become zero-value signals (default route), never an error. Continuation turns reuse the id routed at run start so a run never switches models mid-flight; if that state is lost (recovery), signals are re-extracted deterministically.
Decisions and rejected alternatives
getWorkspaceAiLanguageModel/getWorkspaceAiGatewayProviderOptions/ billing-tier lookup unchanged, so routed turns behave exactly like a user picking that model. This was the materially simpler insertion.beforeTurn, notgetWorkspaceAiLanguageModel. The helper is also used by non-chat paths (base model, compaction) that must stay pinned;beforeTurnis the single choke point with access to messages/body.trackproperties to avoid touching the billing payload shape; telemetry sinks cover observability (noted as a follow-up option).Fact corrections vs the issue prompt
chat-attachment-policy.tsacceptsimage/*); there are no PDF chat attachments. The attachment/long-context rules key on image parts and workspace references (selected items/quotes, which include PDF selections) instead.getWorkspaceAiChatModel(DEFAULT_…)call aroundai-thread.ts:405is the compaction summarizer, not title generation. Titles use a dedicatedAI_THREAD_TITLE_GATEWAY_MODELconstant inai-thread-runtime.ts. Both paths are untouched.Follow-ups noticed
TurnInput.bodyon continuation turns appears to replay the original body, but if it were ever absent, an explicit non-auto run's continuation would resolve to"auto"pre-existing behavior; this PR guards the routing side of that edge but the underlying resolution quirk remains.trackproperties could also carryrequested_model_idfor billing-side analytics.model.requested; with routing, a clearer name would beeffective/usedsplit — left as-is to avoid changing the TCC schema here.How verified
pnpm check,pnpm test, andpnpm verify(check + test + production build) pass locally on every commit.Reviewer pre-empts: no premium bypass (tier enforced at route time, tested); tests are deterministic (pure router, fixed thresholds); fallback covered; non-auto behavior bit-for-bit unchanged (router returns
undefinedfor any non-auto id; only additive telemetry fields differ).🤖 Generated with Claude Code
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.Summary by CodeRabbit