Merge pull request #18 from vmariiechko/release/v1.12.0

Release v1.12.0

Author: vmariiechko

CHANGELOG.md

@@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [1.12.0] - 2026-07-18
+
 ### Changed
 - **Asset `sdp-expectation-notifications`**: notification hygiene revision, grounded in a two-day live investigation (2026-07-17/18, serverless SDP on a non-development-mode target). The v1.11.0 hook notified on every qualifying `flow_progress` event, and the platform emits expectation counts once per microbatch, so one failing expectation notified once per file on a multi-file backlog (measured: 4 notifications for one expectation in one update, counts summing exactly to the single-batch total), re-notified everything on full refresh, and fired within ~70s of every arriving file on a continuous pipeline. Wired to a real channel, that floods.
   - **The hook now throttles: at most one notification per (pipeline, dataset, expectation) per `dq_notify.throttle_seconds` (default 3600).** Two time-aware layers, both fail-open (any state error notifies rather than stays silent): an in-memory last-notified map (always on; timestamp-based, not a seen-set, because continuous pipelines keep one Python process alive and a notify-once set would go silent forever; validated live NOTIFY -> SUPPRESS in window -> NOTIFY after window on a continuous Auto Loader pipeline) and optional durable marker files under `<dq_notify.state_dir>/dq_notify_state/<pipeline>/<dataset>__<expectation>.json` in an existing UC Volume (one human-readable JSON per expectation, overwritten on notify, never growing; validated across full refreshes and process restarts: 12 raw events across two consecutive full refreshes reduced to 2 notifications). State scopes per pipeline automatically from the event's own `origin` (`pipeline_name`/`pipeline_id`, observed present in hook events), so the same code serves any number of pipelines and cannot cross-suppress (validated with two pipelines sharing one state root). The asset deliberately ships no volume resource: state that describes operations should outlive any one bundle, so a new `state_volume_path` prompt (placeholder default = in-memory only) points under a volume the user already owns. `dq_notify.throttle_seconds: "0"` restores the old per-microbatch behavior.

ROADMAP.md

@@ -51,7 +51,7 @@ Modular sub-templates installable via `databricks bundle init <repo-url> --templ
 - `monitoring-sql-warehouse`: Dedicated serverless SQL warehouse tuned for bursty workloads (scheduled Alerts, monitoring queries) with `auto_stop_mins: 1`
 - `sdp-quarantine-pattern`: Lakeflow SDP pipeline demonstrating the inverse-expectations quarantine pattern (drop expectations route bad rows to a separate quarantine table) on `samples.nyctaxi.trips`, with medallion schema separation, a queryable event log, and NULL-safe predicates. Ships a companion agent skill that adapts the pattern to the user's own dataset and self-verifies it
 - `pyspark-test-runner`: Single-file Python wrapper around `pytest` for local PySpark suites that prints a bounded, agent-friendly digest (counts, runnable failing node ids, failures deduplicated by signature) and keeps full output in a log file, so a suite flooding with repetitive failures does not burn a coding agent's context window. Ships a `SKILL.md` for agent integration
-- `sdp-expectation-notifications`: Per-expectation data-quality notification for Lakeflow SDP as a pair: an in-pipeline event hook fires the moment a WARN expectation result is logged, backed by one DABs-managed Alert v2 sweeping the published event log on a schedule (hook delivery is best-effort by platform design; the alert is the guarantee). Ships a companion agent skill that wires the pattern into the user's own pipeline
+- `sdp-expectation-notifications`: Per-expectation data-quality notification for Lakeflow SDP as a pair: an in-pipeline event hook fires the moment a WARN expectation result is logged (since v1.12.0 throttled to at most one notification per expectation per window via a two-layer time-aware de-dup with optional UC Volume marker state, with Slack/Teams/generic webhook formats and secret-scope URL resolution), backed by one DABs-managed Alert v2 sweeping the published event log on a schedule (hook delivery is best-effort by platform design; the alert is the guarantee, one email per state change). Ships a companion agent skill that wires the pattern into the user's own pipelines
 
 **Possible future assets** (ideas for contributors, not commitments):
 - `etl-pipeline`: Medallion-layered Declarative Pipeline with Bronze/Silver layers and DLT expectations

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "databricks-bundle-template"
-version = "1.11.0"
+version = "1.12.0"
 description = "A custom Declarative Automation Bundles template for production-ready, multi-environment data pipeline projects"
 readme = "README.md"
 license = {text = "MIT"}

template/{{.project_name}}/bundle_init_config.json.tmpl

@@ -1,5 +1,5 @@
 {
-  "_template_version": "1.11.0",
+  "_template_version": "1.12.0",
   "_comment": "Configuration values used to generate this project with 'databricks bundle init'",
   "project_name": "{{.project_name}}",
   "environment_setup": "{{.environment_setup}}",