Skip to content

feat(test): add director-agent acl rotation correctness driver#26

Merged
yusufozturk merged 3 commits into
mainfrom
DT-807
Jun 26, 2026
Merged

feat(test): add director-agent acl rotation correctness driver#26
yusufozturk merged 3 commits into
mainfrom
DT-807

Conversation

@erenaslandev

@erenaslandev erenaslandev commented Jun 26, 2026

Copy link
Copy Markdown
Member

Summary by CodeRabbit

  • New Features
    • Added optional YAML acl_rotation configuration for director↔agent ACL rotation correctness cases, including recover/revoke expectations and timing controls.
    • Introduced a dedicated rotation-run path that hot-reloads allowlist changes mid-run without restarting services.
  • Bug Fixes
    • Enforced stricter ACL rotation validation (correct case type, non-empty allowlist entries, non-negative timing, and required subject settings).
    • Improved rotation pass/fail determination and persisted run results.
  • Tests
    • Added unit tests for allowlist patching behavior in nested YAML structures.
    • Added validation tests covering common misconfigurations.

@coderabbitai

coderabbitai Bot commented Jun 26, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

Adds support for director_agent_acl_rotation_correctness with a new acl_rotation config block, type-specific validation, and a dedicated runner path that rewrites director ACLs during execution to verify recover and revoke behavior.

Changes

ACL rotation correctness

Layer / File(s) Summary
Config schema and validation
internal/config/case.go
Adds acl_rotation to TestCase, introduces ACLRotationConfig, defines supported expect values and timing helpers, detects the case type, and validates the block’s presence and constraints.
Runner dispatch and rotation flow
internal/runner/runner.go, internal/runner/acl_rotation_test.go
Dispatches ACL rotation cases to a dedicated flow that stages a writable director config, rewrites acl.allowed_ips, checks receiver counts, saves the run result, and is covered by YAML patch tests.

Sequence Diagram(s)

sequenceDiagram
  participant Runner
  participant Director
  participant Receiver
  participant Store
  Runner->>Director: stage writable ACL config in tmpDir
  Runner->>Director: rewrite acl.allowed_ips
  Director->>Receiver: apply ACL change to delivery
  Runner->>Receiver: sample line counts for recover or revoke
  Runner->>Store: save RunResult
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • namles
  • yusufozturk

Poem

A rabbit hopped through ACLs so spry,
With carrots held high in the morning sky.
"Recover!" it squeaked, then "Revoke!" with cheer,
As director and runner both shifted gear.
Thump! the bench went pitter-pat bright.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: a director-agent ACL rotation correctness driver for tests.
Docstring Coverage ✅ Passed Docstring coverage is 80.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch DT-807

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/config/case.go`:
- Around line 904-915: The ACL rotation validation in tc.ACLRotation needs to
also constrain acl_rotation.to_config to a safe child directory name, since it
is later joined into a config path and could otherwise escape the case config
directory. Add validation alongside the existing
ACLRotationExpect/SettleSeconds/BaselineSeconds checks in the case validation
logic so values like ../other are rejected before the runner uses them.

In `@internal/runner/runner.go`:
- Around line 2419-2437: The rotated-config validation in runner.go only reads
the rotated file but does not verify that it differs from the initial config
solely in acl.allowed_ips. Update the logic around the initialSrc/rotatedSrc
comparison in the runner flow to parse both YAML configs, remove acl.allowed_ips
from each, and compare the normalized documents before any containers start. If
any other field differs, fail early with a clear error so the ACL rotation case
stays isolated to that single setting.
- Around line 2680-2684: The post-revocation check in runner.go is treating a
negative `advanced` value from `rmAfter2.LinesReceived - rmAfter1.LinesReceived`
as a pass because it only compares against `aclLeak`. Update the stop-verdict
logic around `advanced` and `stopped` in the revocation evaluation path to
require a non-negative delta before acceptance, and treat any receiver counter
regression or reset as a failure case. Keep the existing `rmAfter1`, `rmAfter2`,
`finalCount`, and `formatCount` reporting, but ensure the verdict is only true
when `advanced` is at least zero and within the allowed leak threshold.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: b8a89a55-d4dd-4561-9cd1-53366f8c83a0

📥 Commits

Reviewing files that changed from the base of the PR and between ca591c1 and 9ff0503.

📒 Files selected for processing (2)
  • internal/config/case.go
  • internal/runner/runner.go

Comment thread internal/config/case.go
Comment thread internal/runner/runner.go Outdated
Comment thread internal/runner/runner.go Outdated
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 26, 2026

Copy link
Copy Markdown

Deploying pipebench with  Cloudflare Pages  Cloudflare Pages

Latest commit: 3256b96
Status: ✅  Deploy successful!
Preview URL: https://03940665.pipebench.pages.dev
Branch Preview URL: https://dt-807.pipebench.pages.dev

View logs

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/config/case.go`:
- Around line 907-909: The validation in tc.ACLRotation.AllowedIPs only checks
for a non-empty slice, so blank entries like empty strings still pass and later
get written into the director config. Update the case validation in the ACL
rotation path to reject any empty or whitespace-only entries in
tc.ACLRotation.AllowedIPs, and return a validation error from the same
config-checking code that currently handles the non-empty slice check.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: bb0fa736-d27c-4f5d-b0d3-7aef0372e55c

📥 Commits

Reviewing files that changed from the base of the PR and between 9ff0503 and 01c8bcc.

📒 Files selected for processing (3)
  • internal/config/case.go
  • internal/runner/acl_rotation_test.go
  • internal/runner/runner.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/runner/runner.go

Comment thread internal/config/case.go

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
internal/config/acl_rotation_test.go (1)

27-70: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover the remaining validateACLRotation() branches in this table.

This suite skips several new validator paths from internal/config/case.go:886-937: missing acl_rotation, generator-present rejection, and negative settle_seconds / baseline_seconds. Since this file is the dedicated regression surface for that validator, those branches can now drift without any test failing.

Possible additions
 	tests := []struct {
 		name    string
 		mutate  func(tc *TestCase)
 		wantErr string // substring of the expected error, "" = valid
 	}{
 		{name: "valid recover", mutate: func(*TestCase) {}},
+		{
+			name:    "valid revoke",
+			mutate:  func(tc *TestCase) { tc.ACLRotation.Expect = ACLRotationRevoke },
+		},
+		{
+			name:    "missing acl_rotation block",
+			mutate:  func(tc *TestCase) { tc.ACLRotation = nil },
+			wantErr: "requires an `acl_rotation:` block",
+		},
 		{
 			name:    "empty allowed_ips",
 			mutate:  func(tc *TestCase) { tc.ACLRotation.AllowedIPs = nil },
 			wantErr: "allowed_ips must list",
 		},
@@
 		{
+			name: "generator not allowed",
+			mutate: func(tc *TestCase) {
+				tc.Generator = GeneratorConfig{Mode: "otlp"}
+			},
+			wantErr: "must not declare a generator",
+		},
+		{
+			name:    "negative settle_seconds",
+			mutate:  func(tc *TestCase) { tc.ACLRotation.SettleSeconds = -1 },
+			wantErr: "settle_seconds must be >= 0",
+		},
+		{
+			name:    "negative baseline_seconds",
+			mutate:  func(tc *TestCase) { tc.ACLRotation.BaselineSeconds = -1 },
+			wantErr: "baseline_seconds must be >= 0",
+		},
+		{
 			name: "acl_rotation on wrong type",
 			mutate: func(tc *TestCase) {
 				tc.Type = "correctness"
 			},
 			wantErr: "only valid for type",
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@internal/config/acl_rotation_test.go` around lines 27 - 70, Extend the
acl_rotation validator table in acl_rotation_test.go to cover the remaining
validateACLRotation() branches that are currently untested. Add cases for a
missing ACLRotation block, rejection when a generator is present, and negative
settle_seconds/baseline_seconds values, using the existing TestCase mutation
pattern so the checks exercise validateACLRotation() directly.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@internal/config/acl_rotation_test.go`:
- Around line 27-70: Extend the acl_rotation validator table in
acl_rotation_test.go to cover the remaining validateACLRotation() branches that
are currently untested. Add cases for a missing ACLRotation block, rejection
when a generator is present, and negative settle_seconds/baseline_seconds
values, using the existing TestCase mutation pattern so the checks exercise
validateACLRotation() directly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 8d8ee385-5382-44b8-bcb6-4d9e2c19083d

📥 Commits

Reviewing files that changed from the base of the PR and between 01c8bcc and 3256b96.

📒 Files selected for processing (2)
  • internal/config/acl_rotation_test.go
  • internal/config/case.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • internal/config/case.go

@yusufozturk yusufozturk merged commit e5d50b1 into main Jun 26, 2026
5 checks passed
@yusufozturk yusufozturk deleted the DT-807 branch June 26, 2026 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants