Skip to content

Commit 426da66

Browse files
qnbsclaude
andcommitted
feat(proforge): measured supervisor quality gates + review-required label
Wave B · deepens the ProForge supervisor (audit P0): scores were hard-coded constants and thresholds were fixed in code. - Measured scoring: replace the flat 80/85/88/90 pass-scores with a confidence score that scales with how much signal a stage produced relative to manuscript size (findings per ~1000 words), within a pass band. Fail/"suspect" scores now scale with manuscript size too. The supervisor still does NO AI calls — this is heuristic confidence, not editorial quality — but the score now actually varies with the work done instead of always reporting the same number. - Configurable thresholds: new QualityThresholds (largeManuscriptWords + intakeHardGate) on PipelineConfig, defaulted via DEFAULT_QUALITY_THRESHOLDS and threaded from run config → SupervisorAgent + the orchestrator's intake hard gate. - Dashboard: an explicit "Experimental — your review is required" line, so the human-in-the-loop expectation is stated, not just implied in help. i18n: proforge.pipeline.reviewRequired across all 19 locales + bundles. Tests: measured-score-varies-with-findings, configurable-threshold behaviour; existing exact-score assertions relaxed to pass-band ranges. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 4f52b87 commit 426da66

44 files changed

Lines changed: 277 additions & 112 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

components/proForge/ProForgeDashboard.tsx

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -86,11 +86,15 @@ export const ProForgeDashboard: React.FC = () => {
8686
<span className="text-[var(--sc-text-on-accent)] font-bold text-sm">P</span>
8787
</div>
8888
<div>
89-
<div className="flex items-center gap-2">
89+
<div className="flex items-center gap-2 flex-wrap">
9090
<h2 className="text-base font-semibold">{t('proforge.pipeline.title')}</h2>
9191
{/* QNBS-v3: ProForge is an on-by-default but experimental agentic pipeline — label it. */}
9292
<Badge variant="experimental">{t('common.badge.experimental')}</Badge>
9393
</div>
94+
{/* QNBS-v3: PR6 — make the human-in-the-loop expectation explicit, not just implied in help. */}
95+
<p className="text-[11px] text-[var(--sc-warning-fg)] font-medium">
96+
{t('proforge.pipeline.reviewRequired')}
97+
</p>
9498
<p className="text-xs text-[var(--sc-text-tertiary)]">
9599
{currentRun ? currentRun.label : t('proforge.pipeline.noneActive')}
96100
</p>

features/proForge/types.ts

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,8 +95,23 @@ export interface PipelineConfig {
9595
language: string;
9696
/** Max supervisor-triggered retries per stage (0 = no retry, 1 = one retry) */
9797
maxRetries?: 0 | 1;
98+
/** Tunable supervisor quality-gate thresholds (defaults applied when omitted). */
99+
qualityThresholds?: QualityThresholds;
98100
}
99101

102+
/** Tunable thresholds for the heuristic supervisor quality gates. */
103+
export interface QualityThresholds {
104+
/** Word count above which a stage producing zero edits/findings is treated as a fallback signal. */
105+
largeManuscriptWords: number;
106+
/** Intake qualityScore below this hard-fails the run. */
107+
intakeHardGate: number;
108+
}
109+
110+
export const DEFAULT_QUALITY_THRESHOLDS: QualityThresholds = {
111+
largeManuscriptWords: 1000,
112+
intakeHardGate: 30,
113+
};
114+
100115
export const DEFAULT_PIPELINE_CONFIG: PipelineConfig = {
101116
genrePreset: 'general-fiction',
102117
selectedStages: [
@@ -117,6 +132,7 @@ export const DEFAULT_PIPELINE_CONFIG: PipelineConfig = {
117132
autoAcceptThreshold: 0,
118133
language: 'en',
119134
maxRetries: 1,
135+
qualityThresholds: DEFAULT_QUALITY_THRESHOLDS,
120136
};
121137

122138
// ---------------------------------------------------------------------------

graphify-out/GRAPH_REPORT.md

Lines changed: 65 additions & 67 deletions
Large diffs are not rendered by default.

locales/ar/common.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,7 @@
327327
"proforge.loading.publishing": "جارٍ التحضير للعالم…",
328328
"proforge.loading.structural": "جارٍ النظر في شكل قصتك…",
329329
"proforge.pipeline.noneActive": "لا يوجد خط أنابيب نشط",
330+
"proforge.pipeline.reviewRequired": "تجريبي — مراجعتك مطلوبة؛ لا يتم تغيير المخطوطة أبدًا دون موافقتك.",
330331
"proforge.pipeline.title": "خط أنابيب المؤلّف الأمثل",
331332
"proforge.progress.activeStage": "Active Stage",
332333
"proforge.progress.awaitingReviewHint": "⚠️ This stage is awaiting your review. Click the stage button above to review items.",

locales/de/common.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,7 @@
327327
"proforge.loading.publishing": "Macht sich für die Welt bereit…",
328328
"proforge.loading.structural": "Betrachtet die Struktur deiner Geschichte…",
329329
"proforge.pipeline.noneActive": "Keine aktive Pipeline",
330+
"proforge.pipeline.reviewRequired": "Experimentell — deine Prüfung ist erforderlich; das Manuskript wird nie ohne deine Zustimmung geändert.",
330331
"proforge.pipeline.title": "Ultimative Autoren-Pipeline",
331332
"proforge.progress.activeStage": "Aktive Phase",
332333
"proforge.progress.awaitingReviewHint": "⚠️ Diese Phase wartet auf Ihre Prüfung. Klicken Sie oben auf die Phasenschaltfläche, um die Einträge zu prüfen.",

locales/el/common.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,7 @@
327327
"proforge.loading.publishing": "Προετοιμασία για τον κόσμο…",
328328
"proforge.loading.structural": "Κοιτάζοντας τη μορφή της ιστορίας σας…",
329329
"proforge.pipeline.noneActive": "Δεν υπάρχει ενεργός αγωγός",
330+
"proforge.pipeline.reviewRequired": "Πειραματικό — απαιτείται η αξιολόγησή σας· το χειρόγραφο δεν αλλάζει ποτέ χωρίς την έγκρισή σας.",
330331
"proforge.pipeline.title": "Ultimate Author Pipeline",
331332
"proforge.progress.activeStage": "Ενεργό Στάδιο",
332333
"proforge.progress.awaitingReviewHint": "⚠️ Αυτό το στάδιο περιμένει την κριτική σας. Κάντε κλικ στο κουμπί του σταδίου παραπάνω για να ελέγξετε τα στοιχεία.",

locales/en/common.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -531,6 +531,7 @@
531531
"proforge.loading.analytics": "Summing up the run…",
532532
"proforge.pipeline.title": "Ultimate Author Pipeline",
533533
"proforge.pipeline.noneActive": "No active pipeline",
534+
"proforge.pipeline.reviewRequired": "Experimental — your review is required; the manuscript is never changed without your approval.",
534535
"proforge.emptyState.title": "Your manuscript, refined.",
535536
"proforge.emptyState.description": "ProForge reads every chapter, then walks you through editing stage by stage. Nothing gets lost — every change is reversible.",
536537
"proforge.enabledHint": "ProForge Pipeline enabled. Open the Writer view and click the ProForge button in the tools panel to start.",

locales/es/common.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,7 @@
327327
"proforge.loading.publishing": "Preparándose para el mundo…",
328328
"proforge.loading.structural": "Observando la forma de tu historia…",
329329
"proforge.pipeline.noneActive": "Sin pipeline activa",
330+
"proforge.pipeline.reviewRequired": "Experimental: se requiere tu revisión; el manuscrito nunca se modifica sin tu aprobación.",
330331
"proforge.pipeline.title": "Ultimate Author Pipeline",
331332
"proforge.progress.activeStage": "Etapa activa",
332333
"proforge.progress.awaitingReviewHint": "⚠️ Esta etapa está esperando tu revisión. Haz clic en el botón de la etapa de arriba para revisar los elementos.",

locales/eu/common.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,7 @@
327327
"proforge.loading.publishing": "Mundurako prestatzen...",
328328
"proforge.loading.structural": "Zure istorioaren formari begira...",
329329
"proforge.pipeline.noneActive": "Ez dago kanalizazio aktiborik",
330+
"proforge.pipeline.reviewRequired": "Esperimentala — zure berrikuspena beharrezkoa da; eskuizkribua ez da inoiz aldatzen zure oniritzirik gabe.",
330331
"proforge.pipeline.title": "Azken egilearen kanalizazioa",
331332
"proforge.progress.activeStage": "Etapa Aktiboa",
332333
"proforge.progress.awaitingReviewHint": "⚠️ Etapa hau zure iritziaren zain dago. Egin klik goiko faseko botoian elementuak berrikusteko.",

locales/fa/common.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -327,6 +327,7 @@
327327
"proforge.loading.publishing": "آماده شدن برای جهان…",
328328
"proforge.loading.structural": "با نگاه کردن به شکل داستان شما…",
329329
"proforge.pipeline.noneActive": "بدون خط لوله فعال",
330+
"proforge.pipeline.reviewRequired": "آزمایشی — بازبینی شما لازم است؛ نسخه هرگز بدون تأیید شما تغییر نمی‌کند.",
330331
"proforge.pipeline.title": "خط لوله نویسنده نهایی",
331332
"proforge.progress.activeStage": "مرحله فعال",
332333
"proforge.progress.awaitingReviewHint": "⚠️ این مرحله منتظر بررسی شماست. برای بررسی موارد روی دکمه مرحله بالا کلیک کنید.",

0 commit comments

Comments
 (0)