From 7baf239825fcf67db9511f6d85c635acb50f24a8 Mon Sep 17 00:00:00 2001 From: Derek Misler Date: Mon, 2 Mar 2026 15:35:21 -0500 Subject: [PATCH 1/5] feat: add auto issue triage workflow Add a GitHub Actions workflow and cagent agent config that automatically triages bug reports when labeled `kind/bug`. The agent evaluates if the report has enough info, asks for details if not, or implements a fix and opens a draft PR. --- .github/agents/issue-triager.yaml | 153 +++++++++++++++++ .github/workflows/auto-issue-triage.yml | 211 ++++++++++++++++++++++++ 2 files changed, 364 insertions(+) create mode 100644 .github/agents/issue-triager.yaml create mode 100644 .github/workflows/auto-issue-triage.yml diff --git a/.github/agents/issue-triager.yaml b/.github/agents/issue-triager.yaml new file mode 100644 index 000000000..f188f8e56 --- /dev/null +++ b/.github/agents/issue-triager.yaml @@ -0,0 +1,153 @@ +models: + # Sonnet is capable enough for both triage and code fixes + claude-sonnet: + provider: anthropic + model: claude-sonnet-4-5 + max_tokens: 8192 + temperature: 0.1 + +agents: + root: + model: claude-sonnet + description: Triages bug reports and delegates fixes to the fixer sub-agent + sub_agents: + - fixer + instruction: | + You are an issue triage agent. You evaluate GitHub bug reports to determine if they + contain enough information to act on, and if so, delegate to the fixer sub-agent to + implement a fix. + + ## Input + + You receive a prompt containing: + - Issue number, title, body, and labels + - The bug report template for reference + + ## Your workflow + + ### Step 1: Evaluate the issue + + Determine if the issue is actionable by checking: + + 1. **Is it actually a bug?** (not a feature request, question, or support issue) + 2. **Clear description?** Does it explain what's wrong? + 3. **Reproduction steps?** Can we understand how to trigger the bug? + 4. **Expected vs actual behavior?** Do we know what should happen? + + An issue does NOT need to fill in every template field to be actionable. Use judgment: + - A well-written description with clear context can compensate for missing fields + - A stack trace or error message often provides enough to investigate + - Version info is helpful but not always strictly required + + ### Step 2a: If NOT enough info + + If the issue is missing critical information needed to understand or reproduce the bug: + + 1. Use `gh issue comment` to post a polite comment explaining what's missing and + asking for specific details. Be helpful, not bureaucratic. Example: + + ```bash + gh issue comment ISSUE_NUMBER --body "$(cat <<'EOF' + Thanks for reporting this! To help us investigate, could you provide: + + - Steps to reproduce the issue + - The version you're using (`docker agent version`) + - The full error message or stack trace + + This will help us track down the root cause faster. + EOF + )" + ``` + + 2. Add the `status/needs-info` label: + ```bash + gh issue edit ISSUE_NUMBER --add-label "status/needs-info" + ``` + + 3. Output exactly: `RESULT:NEEDS_INFO` + + ### Step 2b: If enough info + + If the issue has enough information to investigate: + + 1. First, explore the codebase to understand the project structure and locate + relevant code related to the bug report + 2. Delegate to the `fixer` sub-agent with a clear description of the bug and + pointers to relevant files/code + 3. When the fixer returns, verify that files were actually modified by listing + changed files. Do NOT rely solely on the fixer's self-reported success: + - If files were actually changed on disk → output exactly: `RESULT:FIXED` + - If no files were changed → output exactly: `RESULT:NO_CHANGES` + + ## Important rules + + - ALWAYS output exactly one of: `RESULT:NEEDS_INFO`, `RESULT:FIXED`, `RESULT:NO_CHANGES` + - The result marker MUST be the LAST line of your output + - Be empathetic in issue comments — these are real users reporting real problems + - Do NOT commit or push any code changes — the workflow handles that + - Do NOT close or reassign issues + + toolsets: + - type: shell + - type: filesystem + - type: think + + fixer: + model: claude-sonnet + description: Investigates bugs and implements fixes in the codebase + instruction: | + You are a bug fixer agent. You receive a bug description from the triager and your + job is to investigate the root cause and implement a fix. + + ## Your workflow + + 1. **Understand the bug**: Read the bug description carefully. Identify what's + going wrong and where in the codebase it might originate. + + 2. **Investigate**: Use the filesystem tools to explore the codebase: + - Read relevant source files + - Trace the code path that triggers the bug + - Look at related tests for context + - Check recent changes to affected files + + 3. **Plan the fix**: Before writing any code, think through: + - What's the root cause? + - What's the minimal change that fixes it? + - Could this fix break anything else? + - Are there existing tests that need updating? + + 4. **Implement**: Make the necessary code changes: + - Keep changes minimal and focused + - Follow existing code style and conventions + - Update or add tests if appropriate + + 5. **Verify**: Run tests and linting to make sure the fix is correct: + ```bash + task test + task lint + ``` + If tests fail, investigate and fix. Do not leave broken tests. + + ## Important rules + + - Do NOT commit or push changes — the workflow handles git operations + - Do NOT modify CI/CD configs, workflows, or unrelated files + - Keep changes minimal — fix the bug, nothing more + - If you cannot determine a fix with confidence, make no changes and explain why + - Always run `task test` and `task lint` before finishing + + toolsets: + - type: filesystem + - type: shell + - type: think + +permissions: + allow: + - shell:cmd=gh issue comment * + - shell:cmd=gh issue edit * + - shell:cmd=gh issue view * + - shell:cmd=gh api * + - shell:cmd=task test* + - shell:cmd=task lint* + - shell:cmd=task build* + - shell:cmd=go * diff --git a/.github/workflows/auto-issue-triage.yml b/.github/workflows/auto-issue-triage.yml new file mode 100644 index 000000000..9baa7e1ed --- /dev/null +++ b/.github/workflows/auto-issue-triage.yml @@ -0,0 +1,211 @@ +name: Auto Issue Triage + +on: + issues: + types: [labeled] + +# Elevated permissions needed for the fix+PR path (commit, push, create PR). +# The needs-info path only uses issues: write, but splitting into separate +# jobs would add complexity without meaningful security benefit since this +# only triggers on maintainer-applied labels. +permissions: + contents: write + issues: write + pull-requests: write + +concurrency: + group: issue-triage-${{ github.event.issue.number }} + cancel-in-progress: true + +jobs: + triage: + if: github.event.label.name == 'kind/bug' + runs-on: ubuntu-latest + timeout-minutes: 15 + env: + HAS_APP_SECRETS: ${{ secrets.CAGENT_REVIEWER_APP_ID != '' }} + + steps: + - name: Checkout repository + uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 + with: + fetch-depth: 1 + + - name: Generate GitHub App token + if: env.HAS_APP_SECRETS == 'true' + id: app-token + continue-on-error: true + uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a # v2 + with: + app_id: ${{ secrets.CAGENT_REVIEWER_APP_ID }} + private_key: ${{ secrets.CAGENT_REVIEWER_APP_PRIVATE_KEY }} + + - name: Construct prompt + id: prompt + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 + with: + script: | + const issue = context.payload.issue; + const labels = issue.labels.map(l => l.name).join(', '); + + const bugTemplate = [ + '## Bug Report Template (for reference)', + '- Describe the bug', + '- Version affected', + '- How To Reproduce (steps)', + '- Expected behavior', + '- Screenshots (optional)', + '- OS and Terminal type', + '- Additional context', + ].join('\n'); + + const prompt = [ + `## Issue #${issue.number}: ${issue.title}`, + '', + `**Labels:** ${labels}`, + `**Author:** ${issue.user.login}`, + `**Created:** ${issue.created_at}`, + '', + '### Issue Body', + '', + issue.body || '(empty)', + '', + '---', + '', + bugTemplate, + '', + '---', + '', + `Triage this bug report. The issue number for gh CLI commands is ${issue.number}.`, + ].join('\n'); + + core.setOutput('text', prompt); + + - name: Run triage agent + id: agent + uses: docker/cagent-action@latest + env: + GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }} + with: + agent: ${{ github.workspace }}/.github/agents/issue-triager.yaml + prompt: ${{ steps.prompt.outputs.text }} + anthropic-api-key: ${{ secrets.ANTHROPIC_API_KEY }} + github-token: ${{ steps.app-token.outputs.token || github.token }} + timeout: 600 + + - name: Parse agent result + id: result + shell: bash + run: | + OUTPUT_FILE="${{ steps.agent.outputs.output-file }}" + if [ ! -f "$OUTPUT_FILE" ]; then + echo "No output file found" + echo "action=none" >> "$GITHUB_OUTPUT" + exit 0 + fi + + CONTENT=$(cat "$OUTPUT_FILE") + echo "--- Agent output ---" + echo "$CONTENT" + echo "--------------------" + + # Check for result markers (search from the end of output) + if echo "$CONTENT" | grep -q "RESULT:NEEDS_INFO"; then + echo "action=needs_info" >> "$GITHUB_OUTPUT" + elif echo "$CONTENT" | grep -q "RESULT:FIXED"; then + echo "action=fixed" >> "$GITHUB_OUTPUT" + elif echo "$CONTENT" | grep -q "RESULT:NO_CHANGES"; then + echo "action=none" >> "$GITHUB_OUTPUT" + else + echo "No recognized result marker found" + echo "action=none" >> "$GITHUB_OUTPUT" + fi + + - name: Check for changes + if: steps.result.outputs.action == 'fixed' + id: changes + shell: bash + run: | + if [ -n "$(git status --porcelain)" ]; then + echo "has_changes=true" >> "$GITHUB_OUTPUT" + else + echo "has_changes=false" >> "$GITHUB_OUTPUT" + fi + + - name: Commit and push fix + if: steps.result.outputs.action == 'fixed' && steps.changes.outputs.has_changes == 'true' + id: push + shell: bash + env: + GITHUB_TOKEN: ${{ steps.app-token.outputs.token || github.token }} + run: | + ISSUE_NUMBER=${{ github.event.issue.number }} + BRANCH_NAME="fix/issue-${ISSUE_NUMBER}" + + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + git checkout -b "$BRANCH_NAME" + git add -A + git commit -m "fix: auto-triage fix for #${ISSUE_NUMBER} + + Automated fix generated by issue triage agent. + Resolves #${ISSUE_NUMBER}" + + git push origin "$BRANCH_NAME" || { + echo "::error::Failed to push branch $BRANCH_NAME" + exit 1 + } + echo "branch=$BRANCH_NAME" >> "$GITHUB_OUTPUT" + + - name: Create draft PR and comment on issue + if: steps.push.outputs.branch != '' + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 + env: + BRANCH_NAME: ${{ steps.push.outputs.branch }} + with: + github-token: ${{ steps.app-token.outputs.token || github.token }} + script: | + const issue = context.payload.issue; + const branch = process.env.BRANCH_NAME; + + // Create draft PR + const pr = await github.rest.pulls.create({ + owner: context.repo.owner, + repo: context.repo.repo, + title: `fix: auto-triage fix for #${issue.number}`, + body: [ + `## Summary`, + ``, + `Automated fix for #${issue.number}.`, + ``, + `> **${issue.title}**`, + ``, + `This PR was generated by the issue triage agent. Please review carefully before merging.`, + ``, + `## Test plan`, + ``, + `- [ ] Review the changes for correctness`, + `- [ ] Verify tests pass in CI`, + `- [ ] Manual testing if applicable`, + ].join('\n'), + head: branch, + base: 'main', + draft: true, + }); + + // Comment on the issue with the PR link + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: issue.number, + body: [ + `I've analyzed this bug report and created an automated fix:`, + ``, + `**Draft PR:** ${pr.data.html_url}`, + ``, + `Please review the changes — this was generated automatically and may need adjustments.`, + ].join('\n'), + }); + + core.info(`Created draft PR #${pr.data.number}: ${pr.data.html_url}`); From 78972bb3d645bed9aeb3ecef7d31cfed653ea4b2 Mon Sep 17 00:00:00 2001 From: Derek Misler Date: Mon, 2 Mar 2026 15:56:52 -0500 Subject: [PATCH 2/5] fix: address PR review feedback and update nightly scanner docs budget Triage workflow: - Add continue-on-error to agent, push, and PR steps - Add fallback notification step so issue authors always get feedback - Parse result marker from last line only (prevents false positives) Nightly scanner: - Always run documentation sub-agent regardless of bug/security findings - Separate issue budgets: 2 bug/security + 1 documentation per run --- .github/agents/nightly-scanner.yaml | 17 ++++++------ .github/workflows/auto-issue-triage.yml | 36 +++++++++++++++++-------- 2 files changed, 34 insertions(+), 19 deletions(-) diff --git a/.github/agents/nightly-scanner.yaml b/.github/agents/nightly-scanner.yaml index 85b1fb95c..8aa4bd6e9 100644 --- a/.github/agents/nightly-scanner.yaml +++ b/.github/agents/nightly-scanner.yaml @@ -53,14 +53,14 @@ agents: - `security` - for security vulnerabilities (HIGHEST PRIORITY) - If fails: log error, continue to bugs - `bugs` - for logic errors, resource leaks, race conditions - - If fails: log error, continue to documentation check - - `documentation` - for missing docs - - ONLY run if BOTH security AND bugs returned `NO_ISSUES` - - (Rationale: documentation issues are lower priority; we avoid noise when real bugs exist) + - If fails: log error, continue to documentation + - `documentation` - for missing docs (ALWAYS run, regardless of other findings) - If fails: log error, continue to reporting 4. Collect findings from each sub-agent (they return text format or `NO_ISSUES`) 5. Filter out any issues where FILE matches patterns from memory - 6. Sort by SEVERITY (critical > high > medium) and select top 1-2 issues + 6. Select findings for the reporter using separate budgets: + - Up to 2 security/bug findings (sorted by SEVERITY: critical > high > medium) + - Up to 1 documentation finding (the highest severity one) 7. Add CATEGORY field to each finding based on source agent: - From security agent → `CATEGORY: security` - From bugs agent → `CATEGORY: bug` @@ -358,9 +358,10 @@ agents: ## Workflow - **ENFORCE: Process at most 2 findings. If you receive more, only process the first 2.** + **ENFORCE: Process at most 2 security/bug findings AND at most 1 documentation finding per run.** + (Maximum 3 issues total: 2 bug/security + 1 documentation.) - For each finding (up to 2 maximum): + For each finding (within the limits above): 1. Check if a similar issue already exists by searching for the same file AND line: ```bash @@ -447,7 +448,7 @@ agents: ## Important - - **STRICT LIMIT: Maximum 2 issues per run** - Stop after creating 2 issues, even if more findings exist + - **STRICT LIMIT: Maximum 2 security/bug issues + 1 documentation issue per run** (3 total max) - Skip duplicates (search by file path AND line number in issue body) - Use exact code snippets from the findings - If creation fails, log FAILED and continue with remaining findings diff --git a/.github/workflows/auto-issue-triage.yml b/.github/workflows/auto-issue-triage.yml index 9baa7e1ed..719f36751 100644 --- a/.github/workflows/auto-issue-triage.yml +++ b/.github/workflows/auto-issue-triage.yml @@ -83,6 +83,7 @@ jobs: - name: Run triage agent id: agent + continue-on-error: true uses: docker/cagent-action@latest env: GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }} @@ -104,20 +105,20 @@ jobs: exit 0 fi - CONTENT=$(cat "$OUTPUT_FILE") echo "--- Agent output ---" - echo "$CONTENT" + cat "$OUTPUT_FILE" echo "--------------------" - # Check for result markers (search from the end of output) - if echo "$CONTENT" | grep -q "RESULT:NEEDS_INFO"; then + # The agent contract requires the result marker on the last line + LAST_LINE=$(tail -n 1 "$OUTPUT_FILE" | tr -d '[:space:]') + if [[ "$LAST_LINE" == "RESULT:NEEDS_INFO" ]]; then echo "action=needs_info" >> "$GITHUB_OUTPUT" - elif echo "$CONTENT" | grep -q "RESULT:FIXED"; then + elif [[ "$LAST_LINE" == "RESULT:FIXED" ]]; then echo "action=fixed" >> "$GITHUB_OUTPUT" - elif echo "$CONTENT" | grep -q "RESULT:NO_CHANGES"; then + elif [[ "$LAST_LINE" == "RESULT:NO_CHANGES" ]]; then echo "action=none" >> "$GITHUB_OUTPUT" else - echo "No recognized result marker found" + echo "::warning::No recognized result marker on last line: $LAST_LINE" echo "action=none" >> "$GITHUB_OUTPUT" fi @@ -135,6 +136,7 @@ jobs: - name: Commit and push fix if: steps.result.outputs.action == 'fixed' && steps.changes.outputs.has_changes == 'true' id: push + continue-on-error: true shell: bash env: GITHUB_TOKEN: ${{ steps.app-token.outputs.token || github.token }} @@ -152,14 +154,13 @@ jobs: Automated fix generated by issue triage agent. Resolves #${ISSUE_NUMBER}" - git push origin "$BRANCH_NAME" || { - echo "::error::Failed to push branch $BRANCH_NAME" - exit 1 - } + git push origin "$BRANCH_NAME" echo "branch=$BRANCH_NAME" >> "$GITHUB_OUTPUT" - name: Create draft PR and comment on issue if: steps.push.outputs.branch != '' + id: pr + continue-on-error: true uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 env: BRANCH_NAME: ${{ steps.push.outputs.branch }} @@ -209,3 +210,16 @@ jobs: }); core.info(`Created draft PR #${pr.data.number}: ${pr.data.html_url}`); + + - name: Notify issue on failure + if: failure() || (steps.result.outputs.action == 'fixed' && steps.changes.outputs.has_changes == 'true' && (steps.push.outcome == 'failure' || steps.pr.outcome == 'failure')) + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7 + with: + github-token: ${{ steps.app-token.outputs.token || github.token }} + script: | + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.payload.issue.number, + body: 'I analyzed this bug report and attempted to create an automated fix, but encountered an error during the process. A maintainer will review this manually.', + }); From 6dc5f1b1f464a4406da132bbfa9073ab9b2df6de Mon Sep 17 00:00:00 2001 From: Derek Misler Date: Mon, 2 Mar 2026 16:15:35 -0500 Subject: [PATCH 3/5] fix: this doesn't need to be a draft --- .github/workflows/auto-issue-triage.yml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/auto-issue-triage.yml b/.github/workflows/auto-issue-triage.yml index 719f36751..ae94e3661 100644 --- a/.github/workflows/auto-issue-triage.yml +++ b/.github/workflows/auto-issue-triage.yml @@ -157,7 +157,7 @@ jobs: git push origin "$BRANCH_NAME" echo "branch=$BRANCH_NAME" >> "$GITHUB_OUTPUT" - - name: Create draft PR and comment on issue + - name: Create PR and comment on issue if: steps.push.outputs.branch != '' id: pr continue-on-error: true @@ -192,7 +192,7 @@ jobs: ].join('\n'), head: branch, base: 'main', - draft: true, + draft: false, }); // Comment on the issue with the PR link @@ -203,13 +203,13 @@ jobs: body: [ `I've analyzed this bug report and created an automated fix:`, ``, - `**Draft PR:** ${pr.data.html_url}`, + `**PR:** ${pr.data.html_url}`, ``, `Please review the changes — this was generated automatically and may need adjustments.`, ].join('\n'), }); - core.info(`Created draft PR #${pr.data.number}: ${pr.data.html_url}`); + core.info(`Created PR #${pr.data.number}: ${pr.data.html_url}`); - name: Notify issue on failure if: failure() || (steps.result.outputs.action == 'fixed' && steps.changes.outputs.has_changes == 'true' && (steps.push.outcome == 'failure' || steps.pr.outcome == 'failure')) From 1799f5a0f0d14bf33f1ad5d39033bf6e5a2d7156 Mon Sep 17 00:00:00 2001 From: Derek Misler Date: Mon, 2 Mar 2026 16:22:06 -0500 Subject: [PATCH 4/5] fix: add /review comment back for bot-opened PRs Auto-review checks org membership, which bots fail. The /review command path (manual-review job) bypasses the org check, so we need to post the comment explicitly. Requires docker/cagent-action#65 to allow the bot. --- .github/workflows/auto-issue-triage.yml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/.github/workflows/auto-issue-triage.yml b/.github/workflows/auto-issue-triage.yml index ae94e3661..f27c1b5c0 100644 --- a/.github/workflows/auto-issue-triage.yml +++ b/.github/workflows/auto-issue-triage.yml @@ -209,6 +209,15 @@ jobs: ].join('\n'), }); + // Trigger AI review (auto-review won't run since the PR author is a bot, + // not an org member — so we use the /review command instead) + await github.rest.issues.createComment({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: pr.data.number, + body: '/review', + }); + core.info(`Created PR #${pr.data.number}: ${pr.data.html_url}`); - name: Notify issue on failure From 0dc488487c7e8b973c464abc31bdf04d004f9f07 Mon Sep 17 00:00:00 2001 From: Derek Misler Date: Mon, 2 Mar 2026 16:24:10 -0500 Subject: [PATCH 5/5] fix: remove stale 'draft' comment from PR creation step --- .github/workflows/auto-issue-triage.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/auto-issue-triage.yml b/.github/workflows/auto-issue-triage.yml index f27c1b5c0..638c95bdf 100644 --- a/.github/workflows/auto-issue-triage.yml +++ b/.github/workflows/auto-issue-triage.yml @@ -170,7 +170,7 @@ jobs: const issue = context.payload.issue; const branch = process.env.BRANCH_NAME; - // Create draft PR + // Create PR const pr = await github.rest.pulls.create({ owner: context.repo.owner, repo: context.repo.repo,