[doc-master] Cross-check CHANGELOG test counts against actual def test_ additions

[akaszubski]

Summary

doc-master writes incorrect test counts in CHANGELOG entries. During the spektiv #951 pipeline (commit c196bfd31, 2026-04-10), doc-master wrote "10 new unit tests" but the actual count was 20 (7 connection + 6 broker + 4 engine wiring + 3 injection regression). The count was likely snapshotted before the remediation cycle added 3 tests and before the final tally.

Implementation Approach

Update agents/doc-master.md to add a test count verification step before writing CHANGELOG entries:

1. Before writing any CHANGELOG entry that mentions test counts, the doc-master MUST enumerate actual def test_ methods added in the current commit's diff:

   git diff HEAD~1 --unified=0 -- 'tests/**/*.py' | grep -c '^\+.*def test_'

   Or for uncommitted changes:

   git diff --unified=0 -- 'tests/**/*.py' | grep -c '^\+.*def test_'

2. Cross-check: If the enumerated count differs from the number the doc-master was about to write, use the enumerated count.

3. Remediation awareness: If the pipeline includes a remediation cycle (STEP 11), the doc-master should be re-invoked with the updated file list (this is already in implement.md STEP 12 — but the re-invocation must also refresh the test count).

Test Scenarios

• doc-master writes CHANGELOG for a PR with 5 new tests → CHANGELOG says "5 new tests" (correct)
• doc-master writes CHANGELOG before remediation adds 3 more tests → doc-master is re-invoked → CHANGELOG says "8 new tests" (correct after refresh)
• doc-master writes CHANGELOG for a PR with 0 new tests → CHANGELOG does not mention test count
• doc-master counts def test_ additions but excludes deleted tests (net count, not gross additions)

Acceptance Criteria

• agents/doc-master.md includes a "Test Count Verification" step before writing CHANGELOG entries
• doc-master enumerates def test_ from the actual git diff, not from agent output or planner estimates
• If remediation cycle occurs, the re-invoked doc-master refreshes the test count
• No CHANGELOG entry contains a test count that differs from the actual def test_ additions in the commit

Origin

Observed during spektiv Issue #951 pipeline (2026-04-10). CIA finding F1. CHANGELOG said "10 new unit tests"; actual was 20.

Related

• [CI-analyst] Route findings by fix target: structural→autonomous-dev, app-code→active repo #739 — Parent pattern: route structural findings to autonomous-dev
• spektiv #951 — Pipeline run where this was observed

Plugin Version: 3.48.0

[akaszubski]

Implemented in 20a7339