Skip to content

Commit a89fc1d

Browse files
erin2722claude
andauthored
chore: fix bot-issues Verify step timeout (raise, trim, sonnet-5) (#984)
The "Verify proposed changes and document sync_models deviations" step was hitting its 15-minute timeout mid-run (Claude still executing when GitHub killed it), so the job failed and no bot-issue PR was created. - Raise timeout-minutes 15 -> 30. - Switch --model claude-opus-4-6 -> claude-sonnet-5 (faster). - Trim the task (the turn/token sinks): - step 4: focus on the field(s) that caused the bail instead of exhaustively re-verifying every field of a complete spec. - step 6: cross-check only the fields being added/changed against sync_models, and look the model up directly instead of fetching + scanning the whole multi-MB LiteLLM catalog. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 7941217 commit a89fc1d

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

.github/workflows/fix-missing-model-bot-issues.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -259,15 +259,15 @@ jobs:
259259
260260
- name: Verify proposed changes and document sync_models deviations
261261
if: steps.fix_initial.outputs.unsupported_count != '0'
262-
timeout-minutes: 15
262+
timeout-minutes: 30
263263
uses: anthropics/claude-code-action@fbda2eb1bdc90d319b8d853f5deb53bca199a7c1 # v1.0.140
264264
with:
265265
anthropic_api_key: ${{ secrets.BRAINTRUST_AGENT_SPEND_API_KEY }}
266266
github_token: ${{ github.token }}
267267

268268
display_report: "true"
269269
claude_args: |
270-
--model claude-opus-4-6
270+
--model claude-sonnet-5
271271
--max-turns 80
272272
--allowedTools "Read,Glob,Grep,LS,WebSearch,WebFetch,Edit,Write"
273273
--disallowedTools "Bash,MultiEdit,Replace,NotebookEditCell,mcp__github__create_issue,mcp__github__create_issue_comment,mcp__github__update_issue,mcp__github__create_pr,mcp__github__create_or_update_file,mcp__github__delete_file,mcp__github_file_ops__commit_files,mcp__github_file_ops__delete_files"
@@ -298,7 +298,7 @@ jobs:
298298
1. Read the unsupported issue numbers file, then process each listed issue independently.
299299
2. For each issue, read its initial result file and original issue body. Use the resolver message to identify exactly which field(s) caused the bail. Focus your research on those fields first.
300300
3. Fetch the official source URLs already listed in the issue first. Search additional official provider docs or APIs only if the listed sources don't provide the missing field.
301-
4. For each model in the issue, build the most complete verified spec you can, not just the missing field. Verify all fields that official sources support:
301+
4. For each model in the issue, focus on the field(s) that caused the bail (identified in step 2). Fill in additional fields only when the official source you are already reading readily provides them — do not open extra sources or spend turns exhaustively re-verifying fields that are unlikely to be wrong. Fields to consider, bail field(s) first:
302302
- `format`, `flavor`, `displayName`
303303
- `parent` when official sources explicitly document a relationship to a stable base alias and that parent model id exists in `model_list.json` — patterns include: dated snapshot → stable alias (e.g. `claude-3-5-sonnet-20241022` → `claude-3-5-sonnet-latest`), `@NNN`/`-vN` versioned snapshots, location-scoped prefixes (`global.`, `us.`, `eu.`, `apac.`), and provider-documented tier variants (e.g. `:free`, `:nitro` on Together AI)
304304
- `available_providers` and provider mapping
@@ -313,7 +313,7 @@ jobs:
313313
- Always include `"global"` first if Google lists the model as available on the global endpoint, followed by per-region codes sorted alphabetically. Match the ordering used by existing vertex entries in `model_list.json`.
314314
- If the model id is not listed on the Google docs page (e.g. preview-only or recently announced), check the model's product page or release notes for explicit region availability. If you still cannot find an official region list, omit `supported_regions` entirely and call this out in the `## Verification` section — do not invent regions.
315315
- Do not copy `supported_regions` from sync_models; sync_models does not track Vertex regions accurately.
316-
6. **Cross-check against sync_models**: Fetch the sync_models reference catalog at `https://raw.githubusercontent.com/BerriAI/litellm/refs/heads/main/litellm/model_prices_and_context_window_backup.json`. For each numeric field in your proposed spec (token limits, pricing), compare it against the sync_models value for the same model. In the `## Verification` section, for every field that differs from sync_models, you must:
316+
6. **Cross-check against sync_models (only the fields you added or changed)**: For just the numeric field(s) you set or corrected in this fix — not the whole spec — compare against the sync_models value for the same model. Look the model up in the sync_models catalog (`https://raw.githubusercontent.com/BerriAI/litellm/refs/heads/main/litellm/model_prices_and_context_window_backup.json`) for those specific fields; do not scan or diff the entire file. In the `## Verification` section, for every changed field that differs from sync_models, you must:
317317
- State the sync_models value explicitly
318318
- State the official source URL that justifies the deviation
319319
- Briefly explain why the official source should be preferred (e.g., sync_models lags, collapses aliases, or has an incorrect mapping)

0 commit comments

Comments
 (0)