feat(testing): add cfn_handler.testing module with replay() and helpers

.github/workflows/ci.yml

@@ -70,7 +70,18 @@ jobs:
         run: uv sync --locked --only-group test
 
       - name: Run tests with coverage
-        run: uv run pytest --cov --cov-report=xml --cov-report=term
+        # `coverage run -m pytest` (NOT `pytest --cov`): the pytest11
+        # entry point in cfn_handler.testing.fixtures causes pytest to
+        # import cfn_handler during plugin collection, BEFORE pytest-cov
+        # attaches its instrumentation. Module-level code in __init__.py
+        # then runs uninstrumented and the report shows artificial 0%
+        # coverage on those lines, dropping aggregate to ~68%. Using
+        # `coverage run` ensures coverage's tracer is active before any
+        # imports. Same pattern as `just test-cov`.
+        run: |
+          uv run coverage run -m pytest
+          uv run coverage xml
+          uv run coverage report
 
       - name: Upload coverage to Codecov
         if: matrix.runner == 'ubuntu-24.04' && matrix.python == '3.12'

README.md

@@ -132,6 +132,45 @@ Working SAM-deployable examples live in `examples/`:
 - `examples/with-physical-id/` — explicit physical resource id (replacement on update).
 - `examples/failing/` — handler that fails, demonstrating FAILED-response semantics.
 
+## Testing your handlers
+
+`cfn-handler` ships first-class testing helpers under `cfn_handler.testing`.
+The dispatch flow runs in-process; no HTTP, no boto3, no moto setup
+required for unit tests:
+
+```python
+from cfn_handler import CustomResource
+from cfn_handler.testing import assert_success, make_event
+
+def test_my_create_handler():
+    resource = CustomResource()
+
+    @resource.create
+    def on_create(event, context):
+        return {"Endpoint": "https://x.example"}
+
+    replay = resource.replay(make_event())
+    assert_success(replay, data={"Endpoint": "https://x.example"})
+```
+
+Available surface:
+
+- `CustomResource.replay(event, context=None)` — execute the dispatch
+  in-process, returning a structured `Replay` (status, data, reason,
+  payload, ...).
+- `make_event(...)`, `make_context(...)` — factories with safe defaults.
+- `assert_success`, `assert_failed`, `assert_deferred` — assertion helpers
+  with informative messages on failure.
+- pytest fixtures `cfn_create_event`, `cfn_update_event`,
+  `cfn_delete_event`, `cfn_lambda_context` — auto-discovered via the
+  `pytest11` entry point; no `pytest_plugins` declaration needed.
+
+For long-running (polled) handlers, the first `replay()` returns
+`Replay(status="DEFERRED")` and mutates the event with marker keys.
+A second `replay()` with the mutated event resumes through the poll
+handler — useful for testing both halves of a polled lifecycle without
+provisioning EventBridge rules.
+
 ## Project status
 
 v1.0.0 — first stable release. Follows [Semantic Versioning](https://semver.org).

docs/ROADMAP.md

@@ -0,0 +1,186 @@
+# Roadmap
+
+This document captures the directional thinking for `cfn-handler`'s
+post-1.x evolution. It's not a commitment — items move freely between
+sections as priorities shift.
+
+OpenSpec changes are the canonical, executable plan. This roadmap is the
+*context* those changes live in.
+
+---
+
+## Vision
+
+`cfn-handler` is a small, focused, zero-dependency library for writing
+CloudFormation Custom Resource Lambda handlers. The library does the
+boilerplate (lifecycle dispatch, presigned-URL response, polling via
+EventBridge, response-status handling) so users focus on the resource
+logic.
+
+The library is **not** a framework. It does not own the user's logging
+stack, idempotency layer, type system, or deployment toolchain. It
+*interoperates* — cleanly — with whatever the user already runs.
+
+---
+
+## Non-goals
+
+These have been considered and explicitly declined. Listed here so we
+don't re-litigate them.
+
+- **CDK construct library** — `aws-cdk.custom-resources.Provider` already
+  occupies that niche, and the audience overlap (CDK + Python + custom
+  resources + cfn-handler) is small. SAM/raw-CFN is the consumer
+  identity.
+- **Step Functions / "Durable" orchestration for >15 min runs** — a
+  non-problem. Polling already re-invokes the Lambda via EventBridge
+  scheduled rules; the actual ceiling is the CFN per-resource timeout
+  and the response-URL TTL, both well above 15 minutes.
+- **CloudFormation macros / Transforms** — antipattern. Account-global
+  registration, hard to debug, requires `CAPABILITY_AUTO_EXPAND`. The
+  modern alternative (CFN Resource Type Registry) is parked separately
+  below.
+- **Async/await handlers** — `async def on_create(...)` would let users
+  use async SDKs natively, but most CFN custom resources are I/O-light.
+  Park unless users ask.
+- **CDK usage docs** — same reasoning as the construct library. Audience
+  too small to justify maintenance overhead. Users who need to call
+  `cfn-handler` from a CDK-deployed Lambda will figure out the obvious
+  shape.
+
+---
+
+## Active priorities (next 3–6 months)
+
+These are scoped, designed, and ready to (or already) ship. Each becomes
+its own OpenSpec change.
+
+### 1. ~~Testing helpers~~ — shipped in `v1.3.0`
+
+Shipped as `cfn_handler.testing`: `replay()`, `Replay` dataclass,
+`make_event` / `make_context` factories, `assert_success` / `assert_failed`
+/ `assert_deferred` helpers, and pytest fixtures auto-discovered via the
+`pytest11` entry point. The legacy `test_mode` / `last_response` surface
+is soft-deprecated (DeprecationWarning) and scheduled for removal in v2.0.
+
+See: archived OpenSpec change `add-testing-helpers` (search
+`openspec/changes/archive/`) and the `testing-helpers` capability spec.
+
+### 2. Better logging — likely `v1.4.0`
+
+**Shape**:
+- A stdlib `logging.Filter` that injects CFN context fields
+  (`StackId`, `LogicalResourceId`, `RequestId`, `RequestType`) into
+  every log record produced during a flow. Works with any logger.
+- A `setup_logging()` opt-in convenience that attaches a JSON formatter
+  and the context filter to the `cfn_handler` logger.
+- Internal lifecycle state-transition logs at INFO:
+  `dispatching:create → handler:returned → response:sent`.
+
+**Powertools interop**: the context filter is a plain `logging.Filter`;
+Powertools users `.addFilter()` it on the Powertools `Logger`. No
+Powertools dep. Document the recipe.
+
+### 3. Optional idempotency module — likely `v1.5.0`
+
+**Shape**:
+- New module `cfn_handler.idempotency` (importable but not in `__all__`).
+- Pluggable backend protocol; the user provides their own DynamoDB
+  table, S3 bucket, or in-memory cache (the last useful only for
+  testing).
+- Usage: `@resource.idempotent(backend=...)` wrapping `@resource.create`
+  etc. Caches the response keyed by `RequestId`; on re-invocation
+  with the same `RequestId`, replays the cached response.
+
+**Powertools interop**: ship a thin adapter that wraps Powertools'
+`@idempotent` so users with that already wired don't need a second
+backend.
+
+**No hard dep on Powertools.**
+
+---
+
+## Future / unscoped
+
+Ideas worth doing, not yet ready.
+
+### Typed events — `v2.0` candidate
+
+`event: dict[str, Any]` everywhere is an anti-pattern given the project's
+"Rust-style error handling" stance. Concrete plan when we get there:
+
+- `cfn_handler.events` module with `CreateEvent`, `UpdateEvent`,
+  `DeleteEvent` `TypedDict` definitions covering the documented CFN
+  custom-resource event shape.
+- Handler decorators preserve the existing `event: dict[str, Any]`
+  signature for backwards compatibility, **and** accept handlers typed
+  with the new TypedDicts via overloads.
+- Powertools interop: structural compatibility with
+  `aws_lambda_powertools.utilities.parser.models.CloudFormationCustomResourceEvent`
+  (pydantic model) — both should satisfy the same `Mapping`-shaped
+  protocol so users can mix-and-match without runtime cost.
+
+This is a breaking change to the `__all__` surface, hence v2.0.
+
+### CFN Resource Type Registry support
+
+The modern CFN private registry mechanism (`AWS::CloudFormation::Resource`)
+lets you publish a resource type with a JSON schema, consumed in templates
+as `MyOrg::MyService::MyResource`. It overlaps with what `cfn-handler`
+does today (Lambda-backed custom resources) but the deployment surface
+and user experience are very different:
+
+- Resource Types are deployed via the CFN registry, not as Lambda
+  functions in the user's account.
+- Schema-validated input/output.
+- Versioning is handled by CFN, not the consumer.
+
+Worth a real exploration session before committing — this is arguably a
+*different product* than `cfn-handler` is today, and might be more sensibly
+shipped as a separate library that shares no code with the runtime.
+
+**Status**: parked, low priority, requires research spike.
+
+---
+
+## Parallel-track work
+
+### `cfn-lint-cfn-handler` plugin (separate repo)
+
+A cfn-lint rule plugin catching `cfn-handler`-specific misconfigurations
+(Lambda timeout too low, polling-using handler missing IAM permissions,
+wrong-region layer ARN against our published manifest).
+
+Lives in its own GitHub repo (`cfn-lint-cfn-handler`) for release-cadence
+independence. Bootstrap context for that repo is in
+`tmp/cfn-lint-plugin-bootstrap.md` (will be deleted once the new repo is
+live).
+
+The headline rule is **W9105: cfn-handler layer ARN doesn't match the
+deployment region**. It consumes the `layer-arns.json` manifest published
+with every `cfn-handler` release. No-one else can write that rule.
+
+---
+
+## Decision log
+
+Things we considered and decided about, kept here for future reference.
+
+| Decision                                                | Date       | Rationale                                                                                       |
+|---------------------------------------------------------|------------|-------------------------------------------------------------------------------------------------|
+| Skip CDK construct                                       | 2026-05-22 | `aws-cdk.custom-resources.Provider` covers it; audience overlap with cfn-handler users is small |
+| Drop SFN/Durable for long-running ops                    | 2026-05-22 | Polling already re-invokes via EventBridge; 15 min Lambda limit is not the actual ceiling       |
+| Hard-pass on CFN macros                                  | 2026-05-22 | Account-global, hard to debug, `CAPABILITY_AUTO_EXPAND` ergonomics                              |
+| cfn-lint plugin in separate repo (not monorepo)          | 2026-05-22 | Independent versioning, simpler CI, avoids workspace-restructure overhead                       |
+| Powertools interop via duck-typing, never as a hard dep  | 2026-05-22 | Zero-dep posture is core; Powertools users get free interop, non-Powertools users unaffected    |
+| Testing helpers ship before logging improvements         | 2026-05-22 | Motivation; testing surface design might inform logging surface design                          |
+
+---
+
+## Updating this document
+
+- New idea? Add it under "Future / unscoped" with a one-paragraph shape.
+- Idea picked up for work? Move to "Active priorities" with a target version.
+- Idea declined? Move to "Non-goals" or the "Decision log" with rationale.
+- Idea shipped? Delete from this doc; the OpenSpec spec captures the
+  durable contract.

justfile

@@ -21,8 +21,16 @@ test-integration:
     uv run pytest tests/integration
 
 # Run the test suite with coverage gate (fails below 95%).
+#
+# Uses `coverage run -m pytest` (NOT `pytest --cov`) so coverage's
+# instrumentation hooks attach BEFORE the pytest11 entry point loads
+# `cfn_handler.testing.fixtures` and transitively imports `cfn_handler`.
+# Otherwise module-level code in __init__.py runs before coverage starts
+# and the report shows artificial 0% coverage on those lines.
 test-cov:
-    uv run pytest --cov --cov-report=term-missing --cov-report=html
+    uv run coverage run -m pytest
+    uv run coverage report --show-missing --fail-under=95
+    uv run coverage html
 
 # Watch tests (re-run on file change). Requires pytest-watcher; install via `uv add --group test pytest-watcher` if not already.
 test-watch:
@@ -138,8 +146,21 @@ test-matrix-arm64: _check-act
 #   4b. ci.yml `lint` job — ruff, ruff-format, mypy strict, pyright strict
 #       (~30s).
 #   4c. examples-lint.yml — cfn-lint over examples/**/template.yaml (~30s).
-#   5. codeql.yml — Python security-and-quality scan (~1-8 min, slower
-#      on first run while CodeQL bundle downloads).
+#   5. codeql.yml — INTENTIONALLY SKIPPED. The CodeQL Action's
+#      post-analysis step calls GitHub's REST API
+#      (/repos/{owner}/{repo}/actions/runs/{run_id}) for telemetry
+#      and status reporting; under `act` the synthesized GITHUB_RUN_ID
+#      doesn't exist on github.com, the call 404s, and the action
+#      classifies the job as JOB_STATUS_CONFIGURATION_ERROR even when
+#      the analysis itself succeeded with zero findings. CodeQL is
+#      validated by the real GH Actions run on every PR; replicating
+#      it here would always report a false-negative failure.
+#      To run CodeQL locally on demand:
+#        act push -W .github/workflows/codeql.yml \
+#          --container-architecture linux/amd64 \
+#          --secret GITHUB_TOKEN="$(gh auth token)"
+#      (Inspect the SARIF in /Users/<you>/.../results/python.sarif —
+#       configuration-error exits with the SARIF generated mean clean.)
 gha-pre-release: _check-act _check-gh-token _check-docker _check-npm
     #!/usr/bin/env bash
     set -uo pipefail
@@ -149,13 +170,13 @@ gha-pre-release: _check-act _check-gh-token _check-docker _check-npm
       --secret GITHUB_TOKEN="$(gh auth token)"
     )
 
-    echo "==> [1/7] secure-workflows.yml — SHA-pin enforcement"
+    echo "==> [1/6] secure-workflows.yml — SHA-pin enforcement"
     act pull_request -W .github/workflows/secure-workflows.yml "${common_flags[@]}" \
         --action-cache-path /tmp/act-cache-secure-workflows \
         || { echo; echo "FAIL: secure-workflows.yml"; exit 1; }
 
     echo
-    echo "==> [2/7] Docker action manifest probe"
+    echo "==> [2/6] Docker action manifest probe"
     # Match `uses: <owner>/<repo>@<sha>` in every workflow file, then for any
     # action that publishes a Docker image at ghcr.io/<owner>/<repo>, verify
     # the SHA resolves to a real image. Currently this is just
@@ -197,7 +218,7 @@ gha-pre-release: _check-act _check-gh-token _check-docker _check-npm
     echo "  (all Docker action images resolve)"
 
     echo
-    echo "==> [3/7] release-please uv.lock validator"
+    echo "==> [3/6] release-please uv.lock validator"
     # Loads release-please's GenericToml updater locally and exercises it
     # against the real uv.lock + the jsonpath in release-please-config.json.
     # `npm ci` is strict-lockfile (matches our uv --locked posture); install
@@ -209,30 +230,34 @@ gha-pre-release: _check-act _check-gh-token _check-docker _check-npm
     ) || { echo; echo "FAIL: release-please uv.lock validator"; exit 1; }
 
     echo
-    echo "==> [4a/7] ci.yml — test matrix (amd64 + arm64 in parallel)"
+    echo "==> [4a/6] ci.yml — test matrix (amd64 + arm64 in parallel)"
     just test-matrix \
         || { echo; echo "FAIL: ci.yml test matrix"; exit 1; }
 
     echo
-    echo "==> [4b/7] ci.yml — lint+typecheck job"
+    echo "==> [4b/6] ci.yml — lint+typecheck job"
     act pull_request -W .github/workflows/ci.yml "${common_flags[@]}" --job lint \
         --action-cache-path /tmp/act-cache-lint \
         || { echo; echo "FAIL: ci.yml lint job"; exit 1; }
 
     echo
-    echo "==> [4c/7] examples-lint.yml — cfn-lint over examples"
+    echo "==> [4c/6] examples-lint.yml — cfn-lint over examples"
     act pull_request -W .github/workflows/examples-lint.yml "${common_flags[@]}" \
         --action-cache-path /tmp/act-cache-examples-lint \
         || { echo; echo "FAIL: examples-lint.yml"; exit 1; }
 
     echo
-    echo "==> [5/7] codeql.yml — Python security analysis"
-    act push -W .github/workflows/codeql.yml "${common_flags[@]}" \
-        --action-cache-path /tmp/act-cache-codeql \
-        || { echo; echo "FAIL: codeql.yml"; exit 1; }
+    echo "==> [5/6] codeql.yml — SKIPPED (act/CodeQL incompatibility)"
+    echo "    The CodeQL Action's post-analysis telemetry call to GitHub's"
+    echo "    REST API 404s under \`act\` because the synthesized GITHUB_RUN_ID"
+    echo "    doesn't exist on github.com, even when the analysis itself"
+    echo "    succeeds with zero findings. CodeQL is validated by the real"
+    echo "    GH Actions run on every PR; see the recipe header comment for"
+    echo "    instructions on running CodeQL locally on demand."
 
     echo
-    echo "OK: all gating jobs passed locally. Safe to merge."
+    echo "OK: all locally-replayable gating jobs passed. Safe to merge."
+    echo "    (CodeQL still gates merge on the actual PR via real GH Actions.)"
 
 # ---- OpenSpec ------------------------------------------------------------
 

openspec/changes/add-testing-helpers/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-05-22