docs: v0.4.0 release — CHANGELOG, README, Hugo docs, EventSLAMet fix (#37)

* docs: update CHANGELOG for v0.4.0 release

* docs: update README architecture for v0.4.0

* docs: update architecture overview for sequential SFN and EventBridge Scheduler

* docs: update AWS architecture for sequential SFN and EventBridge Scheduler SLA

* docs: update SLA docs for EventBridge Scheduler and add execution granularity

* docs: rewrite ASL guide for sequential state machine with EventBridge Scheduler

* docs: fix event type names and add SLA_MET in alerting reference

* docs: fix review findings — stale parallel-branch refs, contradictory state counts, missing INFRA_FAILURE

* fix: rename EventSLAResolved to EventSLAMet to match published event string; fix remaining stale parallel-branch refs in docs

Author: dwsmith1983

CHANGELOG.md

@@ -5,28 +5,47 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased] — Architecture Rewrite
+## [0.4.0] - 2026-03-03
+
+### Added
+
+- **SLA monitoring via EventBridge Scheduler**: one-time Scheduler entries fire `SLA_WARNING` and `SLA_BREACH` events at exact timestamps, replacing the previous parallel-branch polling approach. Schedules auto-delete after firing. On job completion, unfired schedules are cancelled and `SLA_MET` is published.
+- **Sub-daily execution granularity**: pipelines can run at hourly or daily cadence depending on sensor data. When sensors include both `date` and `hour` fields, the framework uses a composite execution date (`2026-03-03T10`). Glue triggers receive `--par_day` and `--par_hour` arguments automatically.
+- **Infrastructure trigger retry**: trigger execution failures (e.g., Glue `ConcurrentRunsExceededException`) retry 4 times with exponential backoff (30s, 60s, 120s, 240s) via Step Functions native Retry. Each failure is logged to the joblog table for audit. This retry budget is separate from `maxRetries` for job failures.
+- **StatusChecker fallback in check-job**: when no terminal joblog entry exists, the orchestrator polls the trigger API directly to determine job status.
 
 ### Changed
 
 - **Declarative validation rules** replace the archetype/trait/evaluator system. Pipeline configs define validation as YAML rules (`exists`, `equals`, `gt`, `gte`, `lt`, `lte`, `age_lt`, `age_gt`) — no custom evaluator code needed.
 - **3 DynamoDB tables** (control, joblog, rerun) replace the single-table design for clearer access patterns and independent scaling.
 - **4 Lambda functions** (stream-router, orchestrator, sla-monitor, watchdog) replace the previous 7+ handlers. The orchestrator is a multi-mode handler covering evaluate, trigger, check-job, and post-run.
-- **~12-state Step Functions workflow** with parallel SLA monitoring branch replaces the 47-state machine.
+- **18-state sequential Step Functions workflow** replaces the 47-state machine. SLA monitoring uses EventBridge Scheduler instead of a parallel branch.
 - **EventBridge events** replace SNS for all alerting and lifecycle notifications.
 - **Reusable Terraform module** — consumers deploy infrastructure without framework code in their repo.
 - **Framework reads DynamoDB only** — external processes push sensor data into the control table.
+- **sla-monitor Lambda** supports 5 modes: `schedule`, `cancel`, `fire-alert`, `calculate`, `reconcile`.
+- **Trigger state** retries infrastructure failures independently of job failure retries (`maxRetries`). Exhausted trigger retries route to SLA cleanup and graceful termination instead of crashing.
 
 ### Removed
 
-- Redis and Postgres storage providers (AWS-only going forward)
+- Redis and Postgres storage providers (AWS-first; GCP and Azure planned after AWS stabilizes)
 - CLI binary (`cmd/interlock`) and HTTP server
 - Archetype, trait, and evaluator subprocess system
 - Local mode (Docker Compose + Redis)
 - Cascade notifications, post-completion drift monitoring, replay support
 - SNS alert sinks, S3 alert sinks
 - cobra, chi, pgx, go-redis, color dependencies
 
+### Fixed
+
+- Pipeline config included in Step Functions execution input, eliminating redundant DynamoDB reads during orchestrator modes (#30)
+- YAML configs converted to JSON before DynamoDB storage (#27)
+- Missing trigger enable variables added to Terraform module (`enable_emr_serverless_trigger`, `enable_sfn_trigger`) (#26)
+- Sensor data `data` map unwrapped in stream-router before trigger condition evaluation (#28)
+- Orchestrator output uses `status` string instead of `passed` boolean (#29)
+- SLA monitor handles relative `:MM` deadline format for hourly pipelines (#35)
+- `check-job` skips non-terminal joblog events (e.g., `infra-trigger-failure`) to prevent infinite polling loops (#36)
+
 ## [0.3.1] - 2026-02-28
 
 ### Fixed
@@ -169,6 +188,7 @@ Initial release of the Interlock STAMP-based safety framework for data pipeline
 
 Released under the [Elastic License 2.0](LICENSE).
 
+[0.4.0]: https://github.com/dwsmith1983/interlock/releases/tag/v0.4.0
 [0.3.1]: https://github.com/dwsmith1983/interlock/releases/tag/v0.3.1
 [0.3.0]: https://github.com/dwsmith1983/interlock/releases/tag/v0.3.0
 [0.2.1]: https://github.com/dwsmith1983/interlock/releases/tag/v0.2.1

README.md

@@ -57,19 +57,17 @@ Supported checks: `exists`, `equals`, `gt`, `gte`, `lt`, `lte`, `age_lt`, `age_g
 └───────────────────┘                                 │
                                           ┌───────────▼──────────────┐
                                           │     Step Functions       │
-                                          │  ~12 states, 2 branches: │
-                                          │  ┌─────────┐ ┌────────┐  │
-                                          │  │ Eval +  │ │  SLA   │  │
-                                          │  │ Trigger │ │Monitor │  │
-                                          │  └─────────┘ └────────┘  │
+                                          │  18 sequential states:   │
+                                          │  Evaluate → Trigger →    │
+                                          │  Poll → SLA → Done       │
                                           └──────────┬───────────────┘
                                                      │
                                     ┌────────────────┼────────────────┐
                                     ▼                ▼                ▼
                               orchestrator     sla-monitor       watchdog
-                              (evaluate,       (calculate,       (stale runs,
-                               trigger,         fire-alert)       missed cron)
-                               check-job,
+                              (evaluate,       (schedule SLA     (stale runs,
+                               trigger,         via EventBridge   missed cron)
+                               check-job,       Scheduler)
                                post-run)
 ```
 
@@ -79,7 +77,7 @@ Supported checks: `exists`, `equals`, `gt`, `gte`, `lt`, `lte`, `age_lt`, `age_g
 |----------|---------|
 | `stream-router` | Routes DynamoDB Stream events, starts Step Function executions |
 | `orchestrator` | Multi-mode handler: evaluate rules, trigger jobs, check status, post-run validation |
-| `sla-monitor` | Calculates warning/breach times, fires SLA events to EventBridge |
+| `sla-monitor` | Schedules SLA alerts via EventBridge Scheduler; cancels on job completion |
 | `watchdog` | Detects stale trigger executions and missed cron schedules |
 
 ### DynamoDB Tables

docs/content/docs/architecture/aws.md

@@ -16,18 +16,15 @@ External processes ──→ DynamoDB control table (SENSOR# writes)
                               ↓
                     stream-router Lambda ──→ Step Functions execution
                                                      │
-                                          ┌──────────┴──────────┐
-                                          ↓                     ↓
-                                    Evaluation            SLA Monitoring
-                                      Branch                 Branch
-                                          │                     │
-                                    orchestrator          sla-monitor
-                                     (Lambda)              (Lambda)
-                                          │
-                              ┌───────────┼──────────┐
-                              ↓           ↓          ↓
-                          Evaluate     Trigger    CheckJob
-                          (rules)      (job)     (poll status)
+                                               Sequential flow:
+                                               Evaluate → Trigger
+                                                     │
+                                               SLA Scheduling
+                                            (EventBridge Scheduler)
+                                                     │
+                                               Poll job status
+                                                     │
+                                               SLA Cleanup → Done
 
 EventBridge schedule ──→ watchdog Lambda ──→ DynamoDB (scan for stale/missed)
                                            ──→ EventBridge (publish alerts)
@@ -101,14 +98,14 @@ Supported trigger types: `http`, `command`, `airflow`, `glue`, `emr`, `emr-serve
 
 ### sla-monitor
 
-Lightweight SLA deadline calculator and alert publisher, invoked by the SLA monitoring branch of the Step Functions state machine. Two modes:
+Manages SLA deadlines using EventBridge Scheduler. Creates one-time schedule entries that fire alerts at exact warning and breach timestamps. Invoked by the Step Functions state machine at two points:
 
-| Mode | Purpose |
+| When | Action |
 |---|---|
-| `calculate` | Compute warning and breach timestamps from deadline, expected duration, and timezone |
-| `fire-alert` | Publish `SLA_WARNING` or `SLA_BREACH` event to EventBridge |
+| After trigger (or validation exhaustion) | Creates two one-time EventBridge Scheduler entries — one for warning, one for breach. Entries auto-delete after firing. |
+| On job completion (or all retries exhausted) | Cancels unfired Scheduler entries. Publishes `SLA_MET` if the job completed before the warning deadline. |
 
-The SLA monitor does not read from DynamoDB -- it receives all necessary data as input from the Step Function.
+When a Scheduler entry fires, it invokes this Lambda to publish the corresponding `SLA_WARNING` or `SLA_BREACH` event to EventBridge.
 
 ### watchdog
 
@@ -121,34 +118,58 @@ See [Watchdog](../watchdog) for the full algorithm.
 
 ## Step Functions State Machine
 
-The state machine orchestrates the pipeline lifecycle using two parallel branches:
+The state machine orchestrates the pipeline lifecycle as a sequential flow of 18 states. SLA monitoring is handled by EventBridge Scheduler rather than a parallel branch.
 
-### Evaluation Branch (7 states)
+### State Flow
+
+```
+InitEvalLoop → Evaluate → IsReady
+  → (passed) Trigger → CheckSLAConfig → ScheduleSLAAlerts → HasTriggerResult
+      → WaitForJob → CheckJob → IsJobDone
+          → (terminal) CheckCancelSLA → CancelSLASchedules → Done
+          → (running) WaitForJob (loop)
+  → (not ready) WaitInterval → IncrementElapsed → CheckWindowExhausted
+      → (window remaining) Evaluate (loop)
+      → (window exhausted) ValidationExhausted → CheckSLAConfig → ... → Done
+```
 
-1. **InitEvalLoop** -- initialize elapsed-seconds counter
-2. **Evaluate** -- invoke orchestrator with `mode=evaluate`, returns `passed` or `not_ready`
-3. **IsReady** -- route on evaluation result: passed goes to Trigger, otherwise WaitInterval
-4. **WaitInterval** -- wait `evaluationIntervalSeconds` before retrying
-5. **IncrementElapsed** -- track total elapsed time using `States.MathAdd`
-6. **CheckWindowExhausted** -- if elapsed >= window, go to ValidationExhausted
-7. **Trigger** -- invoke orchestrator with `mode=trigger`, then poll via WaitForJob/CheckJob/IsJobDone loop
+### Evaluation Loop (7 states)
 
-### SLA Monitoring Branch (5 states)
+1. **InitEvalLoop** — initialize elapsed-seconds counter
+2. **Evaluate** — invoke orchestrator with `mode=evaluate`
+3. **IsReady** — if `passed`, go to Trigger; otherwise WaitInterval
+4. **WaitInterval** — configurable delay between evaluation attempts
+5. **IncrementElapsed** — track total elapsed time via `States.MathAdd`
+6. **CheckWindowExhausted** — if elapsed >= window, go to ValidationExhausted
+7. **ValidationExhausted** — publish `VALIDATION_EXHAUSTED` event
 
-1. **CheckSLAConfig** -- skip branch if no SLA configured
-2. **CalcDeadlines** -- invoke sla-monitor with `mode=calculate`
-3. **WaitForWarning** -- wait until warning timestamp
-4. **FireSLAWarning** -- invoke sla-monitor with `mode=fire-alert, alertType=SLA_WARNING`
-5. **WaitForBreach/FireSLABreach** -- same pattern for breach deadline
+### Trigger and Job Polling (7 states)
+
+1. **Trigger** — invoke orchestrator with `mode=trigger`. Infrastructure failures retry 4 times with exponential backoff (30s, 60s, 120s, 240s)
+2. **CheckSLAConfig** — if SLA configured, schedule alerts; otherwise skip
+3. **ScheduleSLAAlerts** — invoke sla-monitor to create one-time EventBridge Scheduler entries
+4. **HasTriggerResult** — if a job was triggered, poll for completion; otherwise finish
+5. **WaitForJob** — configurable delay between job status checks
+6. **CheckJob** — invoke orchestrator with `mode=check-job`
+7. **IsJobDone** — route on terminal events (success/fail/timeout) or keep polling
+
+### SLA Cleanup (2 states)
+
+1. **CheckCancelSLA** — if SLA was scheduled, cancel unfired entries
+2. **CancelSLASchedules** — invoke sla-monitor to delete Scheduler entries and record final SLA outcome
+
+### Terminal States (2 states)
+
+1. **InfraFailure** — Fail state for unrecoverable infrastructure errors
+2. **Done** — Succeed state
 
 ### Error Handling
 
 Every Task state includes Retry and Catch blocks:
 
-- **Retry**: `IntervalSeconds: 2`, `MaxAttempts: 3`, `BackoffRate: 2` for Lambda service errors
-- **Catch**: `States.ALL` routes to a branch-level Fail state (`EvalBranchFailed` or `SLABranchFailed`)
-
-The top-level Parallel state catches any branch failure and routes to `InfraFailure`.
+- **Default Retry**: `IntervalSeconds: 2`, `MaxAttempts: 3`, `BackoffRate: 2` for Lambda service errors
+- **Trigger Retry**: `IntervalSeconds: 30`, `MaxAttempts: 4`, `BackoffRate: 2` — infrastructure failures (e.g., Glue concurrency limits) get a longer retry budget
+- **Catch**: unrecoverable errors route to `InfraFailure` (Fail state). Trigger exhaustion routes to `CheckCancelSLA` for graceful SLA cleanup before termination.
 
 ### ARN Substitution
 

docs/content/docs/architecture/overview.md

@@ -106,56 +106,45 @@ Each pipeline execution follows this lifecycle, orchestrated by Step Functions:
 ```
 Sensor data arrives
         │
-        ↓
+        ▼
 stream-router starts SFN
         │
+        ▼
+   Evaluation Loop
+   (Evaluate → Wait → Re-evaluate)
+        │
    ┌────┴────┐
-   ↓         ↓
-Evaluation  SLA Monitor
-   Loop     (parallel)
-   │             │
-   │      CalcDeadlines
-   │             │
-   │      WaitForWarning
-   │             │
-   │      FireSLAWarning
-   │             │
-   │      WaitForBreach
-   │             │
-   │      FireSLABreach
+   ▼         ▼
+ Passed    Window Exhausted
+   │         │
+   ▼         ▼
+ Trigger   VALIDATION_EXHAUSTED
+   │         │
+   ├─────────┘
+   ▼
+CheckSLA → ScheduleSLAAlerts (EventBridge Scheduler)
+   │
+   ▼
+ WaitForJob → CheckJob → IsJobDone
    │
-   ├─→ Evaluate (validation rules)
-   │       │
-   │   ┌───┴───┐
-   │   ↓       ↓
-   │  Ready  Not Ready
-   │   │       │
-   │   │    Wait → re-evaluate
-   │   │       │
-   │   │    Window exhausted?
-   │   │       │
-   │   │    VALIDATION_EXHAUSTED
-   │   │
-   │   ↓
-   │  Trigger job
-   │   │
-   │  Wait → CheckJob
-   │   │
-   │  success / fail / timeout
+   ▼
+CheckCancelSLA → CancelSLASchedules
    │
-   ↓
-Reconcile (merge branch results)
+   ▼
+ Done
 ```
 
 ### State Summary
 
-The Step Functions state machine uses two parallel branches:
+The Step Functions state machine uses 18 sequential states:
+
+1. **Evaluation loop** — evaluates validation rules at a configurable interval. When all rules pass, triggers the job. If the evaluation window expires, publishes `VALIDATION_EXHAUSTED`.
 
-1. **Evaluation branch** -- loops on Evaluate, checking validation rules at a configurable interval. When all rules pass, triggers the job and polls for completion. Exits on success, failure, timeout, or window exhaustion.
+2. **SLA scheduling** — after trigger (or validation exhaustion), creates one-time EventBridge Scheduler entries for SLA warning and breach deadlines. The Scheduler fires alerts independently at exact timestamps.
 
-2. **SLA monitoring branch** -- calculates warning and breach timestamps from the SLA config, waits until each deadline, and publishes alerts to EventBridge. Skipped entirely if no SLA is configured.
+3. **Job polling** — polls the triggered job for completion via `check-job` mode. Terminal events (success, fail, timeout) proceed to SLA cleanup.
 
-Both branches run concurrently. The evaluation branch handles the actual pipeline lifecycle; the SLA branch is purely observational and never stops execution.
+4. **SLA cleanup** — cancels unfired SLA Scheduler entries and publishes `SLA_MET` if the job completed before the warning deadline.
 
 ## Event System
 
@@ -170,6 +159,7 @@ All lifecycle events are published to a custom EventBridge event bus. This repla
 | `JOB_FAILED` | Triggered job failed |
 | `SLA_WARNING` | SLA warning deadline reached |
 | `SLA_BREACH` | SLA breach deadline reached |
+| `SLA_MET` | Job completed before SLA warning deadline |
 | `RETRY_EXHAUSTED` | All retry attempts consumed |
 | `SFN_TIMEOUT` | Step Function execution timed out (watchdog) |
 | `SCHEDULE_MISSED` | Cron schedule passed without a trigger (watchdog) |
@@ -205,6 +195,10 @@ When a triggered job fails or times out, the stream-router processes the `JOB#`
 
 Re-run executions use a unique name (`{pipeline}-{schedule}-{date}-rerun-{attempt}`) to avoid Step Function dedup collisions.
 
+### Infrastructure Trigger Retry
+
+When the trigger execution itself fails (e.g., Glue `ConcurrentRunsExceededException`), the orchestrator logs the failure to the joblog table and returns a Lambda error. Step Functions retries the trigger 4 times with exponential backoff (30s, 60s, 120s, 240s). This retry budget is separate from `maxRetries`. If all attempts fail, the execution routes to SLA cleanup and terminates gracefully.
+
 ## Post-Run Validation
 
 Pipelines can define optional post-run validation rules that are evaluated after the triggered job completes. These use the same declarative rule syntax as pre-trigger validation: