From 3095d3b929ea931152b1d2b2839751528770e161 Mon Sep 17 00:00:00 2001 From: qnbs <155236708+qnbs@users.noreply.github.com> Date: Wed, 24 Jun 2026 17:27:20 +0200 Subject: [PATCH 1/3] =?UTF-8?q?docs(deepsource):=20codify=20always-trigger?= =?UTF-8?q?=20AI=20review=20on=20every=20PR=20(=C2=A70a)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit DeepSource runs two layers: static analysis (auto on push) and AI Review (on-demand for this team). The runbook now mandates triggering `@deepsourcebot review` on every PR + after each code-changing push, and the §1 table / §2 loop / Consequence note are corrected to distinguish the auto static re-run from the manual AI-review trigger. Co-Authored-By: Claude Opus 4.8 --- docs/DEEPSOURCE-REVIEW-LOOP.md | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/docs/DEEPSOURCE-REVIEW-LOOP.md b/docs/DEEPSOURCE-REVIEW-LOOP.md index cd659e2e..e0a75b87 100644 --- a/docs/DEEPSOURCE-REVIEW-LOOP.md +++ b/docs/DEEPSOURCE-REVIEW-LOOP.md @@ -15,11 +15,25 @@ **without being asked**. Goal state: **DeepSource checks green** (or every finding fixed/justified) with the rest of CI green. -## 1. How DeepSource differs from the CodeAnt loop (read first) +## 0a. ALWAYS trigger the AI review on every PR (standing rule) + +DeepSource runs **two** layers, and only one is automatic: + +- **Static analysis** (per-analyzer check-runs: `DeepSource: JavaScript / Rust / Docker / CSS`) — + **automatic** on every push. +- **AI Review** — **on-demand for this team.** It does **not** run on its own; you must trigger it. + +> **So on EVERY PR (and after each push that changes code), post:** +> ```bash +> gh pr comment --body "@deepsourcebot review" +> ``` +> Then run the correction loop (§2) over whatever the AI review surfaces, in addition to the automatic +> static-analysis findings. A PR is review-quiescent only when **both** layers are clean. Trigger it +> right after opening the PR and again after every code-changing push (docs-only pushes don't need it). | Aspect | CodeAnt | DeepSource | |---|---|---| -| Trigger | manual `@codeant-ai review` per push | **automatic on every push** — no comment needed | +| Trigger | manual `@codeant-ai review` per push | **static**: auto on every push · **AI review**: on-demand → `@deepsourcebot review` on every PR (§0a) | | Where findings appear | GitHub **review threads** (resolvable) | **check-run annotations** (per file/line) + the DeepSource **dashboard**; *not* review threads | | Resolution mechanism | reply + `resolveReviewThread` (GraphQL) | **fix the code** (check goes green) · `# skipcq` inline · or "Ignore" in the dashboard | | Suppression token | `// biome-ignore` | `# skipcq: ` / `// skipcq: ` | @@ -27,8 +41,9 @@ | Autofix | — | **dashboard-driven** — opens its own PR (review it like any PR) | **Consequence:** there is **no `resolveReviewThread` step** here. You make a check green by fixing the -code (preferred), by a justified `# skipcq`, or by ignoring it in the dashboard. Because DeepSource -re-runs on every push, the loop's "re-trigger" step is **automatic**. +code (preferred), by a justified `# skipcq`, or by ignoring it in the dashboard. The **static** +re-analysis re-runs automatically on push — but the **AI review is on-demand**, so re-trigger it with +`@deepsourcebot review` after each code-changing push (§0a). ## 2. The Iron Rule — loop until quiescent @@ -45,7 +60,7 @@ caused by the fix (a "wave"). Handle each wave like the first. │ 4. Update tests + i18n + docs (lockstep) │ │ 5. suppressions + lint + typecheck + vitest │ │ 6. Commit + push (one wave = one commit) │ - │ 7. DeepSource re-runs AUTOMATICALLY on push │ + │ 7. static auto-reruns; re-trigger AI (§0a) │ └───────────────┬─────────────────────────────┘ │ new findings? ┌── yes ──┘ └── no ──┐ From f9e02f6ee5418d99ec5f278479e3b7e89a05e8fd Mon Sep 17 00:00:00 2001 From: qnbs <155236708+qnbs@users.noreply.github.com> Date: Wed, 24 Jun 2026 17:32:42 +0200 Subject: [PATCH 2/3] docs(deepsource): AI-review prerequisite + best-effort posture (docs-grounded) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Per the DeepSource docs, AI review needs AI Agents enabled (dashboard → Policies → AI → "Enable AI Agents"); inline AI findings via Settings → Quality Gates. Observed 2026-06-24: `@deepsourcebot review` produced no response on #231/#232/#233 (AI Agents off / not on the OSS tier), so the runbook now treats the AI review as BEST-EFFORT — always trigger it, but gate merges on static-analysis quiescence (the reliable layer), same posture as the CodeAnt-unresponsive rule. Also restores the §1 header (lost in the §0a insertion) and logs the repo-wide triage + AI-review observation in §11. Co-Authored-By: Claude Opus 4.8 --- docs/DEEPSOURCE-REVIEW-LOOP.md | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/docs/DEEPSOURCE-REVIEW-LOOP.md b/docs/DEEPSOURCE-REVIEW-LOOP.md index e0a75b87..2610ac8e 100644 --- a/docs/DEEPSOURCE-REVIEW-LOOP.md +++ b/docs/DEEPSOURCE-REVIEW-LOOP.md @@ -31,6 +31,22 @@ DeepSource runs **two** layers, and only one is automatic: > static-analysis findings. A PR is review-quiescent only when **both** layers are clean. Trigger it > right after opening the PR and again after every code-changing push (docs-only pushes don't need it). +### Prerequisite + reality (DeepSource docs) + +The AI review only runs if **AI Agents is enabled**: dashboard → **Policies → AI → "Enable AI Agents"** +(on by default for new accounts; existing accounts must switch it on). Inline AI findings (vs. only the +grade summary) are toggled in **Settings → Quality Gates → inline review comments**. + +**Observed (2026-06-24):** even after triggering `@deepsourcebot review` on three PRs, the AI review +produced **no response** (no post-trigger `deepsource-io` comment; the AI-only CSS analyzer stayed +`skipped`) — AI Agents is off or the OSS/free tier doesn't serve it. **So treat the AI review as +best-effort, NOT a merge gate:** always trigger it, but if it doesn't respond, proceed on +**static-analysis quiescence** (same posture as the CodeAnt-unresponsive rule). The static layer +(JavaScript/Rust/Docker check-runs + dashboard issues) is the reliable one and already covers +security / bug-risk / anti-pattern. + +## 1. How DeepSource differs from the CodeAnt loop (read first) + | Aspect | CodeAnt | DeepSource | |---|---|---| | Trigger | manual `@codeant-ai review` per push | **static**: auto on every push · **AI review**: on-demand → `@deepsourcebot review` on every PR (§0a) | @@ -203,6 +219,14 @@ GitHub App resumes auto-reviewing, run **both** loops: CodeAnt for narrative/AI beyond the single `javascript` analyzer declared in the toml. Findings are check annotations, not review threads. Autofix enabled by the maintainer (dashboard-driven). First full-codebase pass will land when the config reaches `main`. +- **2026-06-24** — Repo-wide triage done off the **static** layer (dashboard categories via WebFetch): + Security clean (JS-0440 dashboard-ignored, reviewed-safe), ~1700 anti-patterns + bug-risk almost all + rule-ignored as deliberate-convention/Biome/strict-TS/test false-positives (void/any/non-null/console/ + async-no-await/…), Performance + Documentation **0**. Genuine fixes: ecoModeService boolean (#230), + Storybook rules-of-hooks (#231), PDF-iframe sandbox (#232). **AI review never responded** to + `@deepsourcebot review` on #231/#232/#233 (no post-trigger comment; CSS analyzer stayed skipped) → + AI Agents off or not on the OSS tier. Adopted the **best-effort** posture above (trigger always, + gate on static). --- From 5d94c5e2958a06c04deaf3e59189388b7fd3c8db Mon Sep 17 00:00:00 2001 From: qnbs <155236708+qnbs@users.noreply.github.com> Date: Wed, 24 Jun 2026 20:33:48 +0200 Subject: [PATCH 3/3] =?UTF-8?q?docs(deepsource):=20remove=20AI-review=20tr?= =?UTF-8?q?igger=20rule=20=E2=80=94=20paid=20feature,=20unavailable=20on?= =?UTF-8?q?=20free=20tier?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit AI Review (@deepsourcebot review) requires AI Agents (dashboard → Policies → AI), a paid/team feature absent on the free OSS plan — verified zero response on #231/#232/#233. Reverse the prior always-trigger rule: do not post the trigger; gate merges on the static-analysis layer's quiescence (auto-runs on every push). Co-Authored-By: Claude Opus 4.8 --- docs/DEEPSOURCE-REVIEW-LOOP.md | 52 +++++++++++++--------------------- 1 file changed, 20 insertions(+), 32 deletions(-) diff --git a/docs/DEEPSOURCE-REVIEW-LOOP.md b/docs/DEEPSOURCE-REVIEW-LOOP.md index 2610ac8e..a2ee1709 100644 --- a/docs/DEEPSOURCE-REVIEW-LOOP.md +++ b/docs/DEEPSOURCE-REVIEW-LOOP.md @@ -15,41 +15,28 @@ **without being asked**. Goal state: **DeepSource checks green** (or every finding fixed/justified) with the rest of CI green. -## 0a. ALWAYS trigger the AI review on every PR (standing rule) +## 0a. AI Review is NOT available on this account — do not trigger it -DeepSource runs **two** layers, and only one is automatic: +DeepSource has **two** layers: -- **Static analysis** (per-analyzer check-runs: `DeepSource: JavaScript / Rust / Docker / CSS`) — - **automatic** on every push. -- **AI Review** — **on-demand for this team.** It does **not** run on its own; you must trigger it. +- **Static analysis** (per-analyzer check-runs `DeepSource: JavaScript / Rust / Docker`; `CSS` is + AI-only) — **automatic on every push, free, reliable.** This is the **operative** layer; the entire + loop below runs on it. +- **AI Review** (LLM-based, triggered by `@deepsourcebot review`) — requires **AI Agents** + (dashboard → Policies → AI), a **paid / team feature not on the free OSS tier.** -> **So on EVERY PR (and after each push that changes code), post:** -> ```bash -> gh pr comment --body "@deepsourcebot review" -> ``` -> Then run the correction loop (§2) over whatever the AI review surfaces, in addition to the automatic -> static-analysis findings. A PR is review-quiescent only when **both** layers are clean. Trigger it -> right after opening the PR and again after every code-changing push (docs-only pushes don't need it). - -### Prerequisite + reality (DeepSource docs) - -The AI review only runs if **AI Agents is enabled**: dashboard → **Policies → AI → "Enable AI Agents"** -(on by default for new accounts; existing accounts must switch it on). Inline AI findings (vs. only the -grade summary) are toggled in **Settings → Quality Gates → inline review comments**. - -**Observed (2026-06-24):** even after triggering `@deepsourcebot review` on three PRs, the AI review -produced **no response** (no post-trigger `deepsource-io` comment; the AI-only CSS analyzer stayed -`skipped`) — AI Agents is off or the OSS/free tier doesn't serve it. **So treat the AI review as -best-effort, NOT a merge gate:** always trigger it, but if it doesn't respond, proceed on -**static-analysis quiescence** (same posture as the CodeAnt-unresponsive rule). The static layer -(JavaScript/Rust/Docker check-runs + dashboard issues) is the reliable one and already covers -security / bug-risk / anti-pattern. +> **Verified 2026-06-24:** triggering `@deepsourcebot review` on three PRs produced **zero response** +> (no comment; the AI-only CSS analyzer stayed `skipped`), and the "Enable AI Agents" toggle is not +> even present on the free plan. **So do NOT post `@deepsourcebot review`** — it is a no-op that only +> adds a dead comment. Run the correction loop on the **static-analysis** findings + the dashboard +> categories. **If the account ever upgrades** to include AI Review, re-introduce the per-PR trigger +> and fold its findings into the loop. ## 1. How DeepSource differs from the CodeAnt loop (read first) | Aspect | CodeAnt | DeepSource | |---|---|---| -| Trigger | manual `@codeant-ai review` per push | **static**: auto on every push · **AI review**: on-demand → `@deepsourcebot review` on every PR (§0a) | +| Trigger | manual `@codeant-ai review` per push | **static**: auto on every push (the operative layer) · **AI review**: paid feature, unavailable on the free tier — not triggered (§0a) | | Where findings appear | GitHub **review threads** (resolvable) | **check-run annotations** (per file/line) + the DeepSource **dashboard**; *not* review threads | | Resolution mechanism | reply + `resolveReviewThread` (GraphQL) | **fix the code** (check goes green) · `# skipcq` inline · or "Ignore" in the dashboard | | Suppression token | `// biome-ignore` | `# skipcq: ` / `// skipcq: ` | @@ -58,8 +45,8 @@ security / bug-risk / anti-pattern. **Consequence:** there is **no `resolveReviewThread` step** here. You make a check green by fixing the code (preferred), by a justified `# skipcq`, or by ignoring it in the dashboard. The **static** -re-analysis re-runs automatically on push — but the **AI review is on-demand**, so re-trigger it with -`@deepsourcebot review` after each code-changing push (§0a). +re-analysis re-runs **automatically on every push** — no trigger comment is needed (and the on-demand +AI review is a paid feature that is unavailable on this account; see §0a). ## 2. The Iron Rule — loop until quiescent @@ -76,7 +63,7 @@ caused by the fix (a "wave"). Handle each wave like the first. │ 4. Update tests + i18n + docs (lockstep) │ │ 5. suppressions + lint + typecheck + vitest │ │ 6. Commit + push (one wave = one commit) │ - │ 7. static auto-reruns; re-trigger AI (§0a) │ + │ 7. static re-runs AUTOMATICALLY on push │ └───────────────┬─────────────────────────────┘ │ new findings? ┌── yes ──┘ └── no ──┐ @@ -225,8 +212,9 @@ GitHub App resumes auto-reviewing, run **both** loops: CodeAnt for narrative/AI async-no-await/…), Performance + Documentation **0**. Genuine fixes: ecoModeService boolean (#230), Storybook rules-of-hooks (#231), PDF-iframe sandbox (#232). **AI review never responded** to `@deepsourcebot review` on #231/#232/#233 (no post-trigger comment; CSS analyzer stayed skipped) → - AI Agents off or not on the OSS tier. Adopted the **best-effort** posture above (trigger always, - gate on static). + AI Agents off / not on the OSS free tier. **Conclusion (#233):** AI Review is a paid feature + unavailable here — **stop triggering it** and gate merges on the **static** layer's quiescence + (§0a). Re-introduce the per-PR trigger only if the account is upgraded to include AI Agents. ---