R1.5 Preference categories and double opt-in

.env.example

@@ -1,5 +1,8 @@
 AWS_REGION=us-east-1
 PUBLIC_BASE_URL=http://localhost:3000
+# Public base URL of the client site page that receives double opt-in confirm
+# links (Relay appends ?token=...). Defaults to PUBLIC_BASE_URL when empty.
+SUBSCRIPTION_CONFIRM_BASE_URL=
 DEFAULT_FROM_EMAIL=newsletter@example.com
 DATABASE_URL=sqlite:///db.sqlite3
 DEBUG=True

docs/api.md

@@ -127,16 +127,147 @@ Use `PUT` to replace the audience tag set. Use single-tag `POST` and `DELETE` fo
 ```text
 POST /api/subscriptions/subscribe
 POST /api/subscriptions/unsubscribe
+POST /api/subscriptions/request-verification
+POST /api/subscriptions/confirm
 ```
 
-Subscribe creates the contact if needed and marks the client-scoped subscription subscribed.
+### Canonical preference categories
+
+Subscription preferences use one canonical category vocabulary. A canonical
+category name is stored as the `CategoryPreference` tag, so send-time
+suppression works through the same tag path as free-form `category_tag`
+values.
+
+| Category | Opt-out | Meaning |
+|---|---|---|
+| `newsletter` | yes | Newsletter and marketing digest sends. |
+| `events` | yes | Event announcements, reminders, and recaps. |
+| `courses` | yes | Course announcements and cohort communication. |
+| `product` | yes | Product updates and release notes. |
+| `transactional` | never | Account, security, and delivery-critical messages. Always on. |
+
+The `transactional` category is always on: a contact cannot opt out of it, and
+a transactional send tagged with `category_tag: "transactional"` is never
+suppressed by the category check.
+
+### Subscribe and unsubscribe with a category
+
+Subscribe creates the contact if needed and marks the client-scoped subscription subscribed. The optional `category` field also enables the canonical category preference for the scoped contact:
+
+```json
+{
+  "email": "subscriber@example.com",
+  "audience": "datatalks-club",
+  "client": "dtc-newsletter",
+  "category": "events"
+}
+```
 
 Unsubscribe accepts `scope` values:
 
 - `client`: unsubscribe from one client.
 - `audience`: unsubscribe from the whole audience.
 - `global`: unsubscribe from all marketing email managed by Datamailer.
 
+The optional `category` field also disables the canonical category preference for the scoped contact:
+
+```json
+{
+  "email": "subscriber@example.com",
+  "audience": "datatalks-club",
+  "client": "dtc-newsletter",
+  "scope": "client",
+  "category": "events",
+  "reason": "user_requested"
+}
+```
+
+Validation rules for `category` on both endpoints:
+
+- Unknown values are rejected with `400` and `{"category": "unknown"}`.
+- `unsubscribe` with `category: "transactional"` is rejected with `400` and
+  `{"category": "transactional_cannot_be_disabled"}`.
+
+Requests without `category` behave exactly as before: no preference row is
+created or changed.
+
+### Double opt-in
+
+Double opt-in proves a recipient wants a category before it is enabled. The
+flow is stateless on the Datamailer side: the confirmation token is signed and
+expiring, carries only ids and the category name (never a raw email), and
+requires no database row.
+
+1. The client calls `POST /api/subscriptions/request-verification`:
+
+```json
+{
+  "email": "subscriber@example.com",
+  "audience": "datatalks-club",
+  "client": "dtc-newsletter",
+  "category": "newsletter",
+  "template_key": "newsletter-double-opt-in"
+}
+```
+
+   Datamailer validates the scope, rejects `category: "transactional"` with
+   `400` and `{"category": "transactional_not_allowed"}`, and fails closed
+   with `404` and `{"template_key": "not_found"}` when the template is
+   missing, inactive, or owned by another client. It then enqueues one
+   transactional message through that template. The message context carries
+   `confirm_url`, `verification_token`, and `category`; `confirm_url` is built
+   from the `SUBSCRIPTION_CONFIRM_BASE_URL` setting plus the token, so the
+   link lands on the client site's public confirm page:
+
+```text
+{SUBSCRIPTION_CONFIRM_BASE_URL}?token=<opaque signed token>
+```
+
+   The response repeats the request scope without the token:
+
+```json
+{
+  "status": "verification_requested",
+  "email": "subscriber@example.com",
+  "audience": "datatalks-club",
+  "client": "dtc-newsletter",
+  "category": "newsletter",
+  "template_key": "newsletter-double-opt-in"
+}
+```
+
+2. The recipient opens `confirm_url` on the client site. The site extracts the
+   `token` query parameter and calls
+   `POST /api/subscriptions/confirm` with its own API key:
+
+```json
+{
+  "token": "<opaque signed token from confirm_url>"
+}
+```
+
+   Datamailer validates the signature, expiry (48 hours), and that the token
+   was issued for the authenticated client, then enables the category
+   preference (creating it when needed) and returns the resulting preference
+   state:
+
+```json
+{
+  "email": "subscriber@example.com",
+  "audience": "datatalks-club",
+  "client": "dtc-newsletter",
+  "category": {
+    "tag": "newsletter",
+    "label": "Newsletter",
+    "enabled": true
+  }
+}
+```
+
+   Confirmation is idempotent: repeating it returns the same state. A
+   malformed, tampered, expired, or foreign-client token is rejected with
+   `400` and `{"token": "invalid"}`.
+
 ## Import and Export APIs
 
 ```text
@@ -469,6 +600,26 @@ Local transactional send examples require the Datamailer server to be started wi
 
 Transactional sends do not require marketing subscription, but they are blocked for hard bounces and complaints.
 
+A send with a `category_tag` is also suppressed when the contact opted out of
+that category (the canonical categories from the Subscription APIs section use
+the same tag path). The send is rejected with `409`, the message is stored as
+`skipped`, and the error payload names the reason and confirms the
+suppression:
+
+```json
+{
+  "error": {
+    "code": "transactional_suppressed",
+    "message": "Contact is hard-suppressed for transactional email.",
+    "reason": "category_unsubscribe",
+    "suppressed": true
+  }
+}
+```
+
+A send with `category_tag: "transactional"` is never suppressed by the
+category check.
+
 ### Transactional Message Status
 
 ```text

mailing/services/api.py

@@ -35,6 +35,7 @@
 from mailing.services.api_errors import ApiValidationError
 from mailing.services.campaign_sender import render_campaign_message, send_campaign_test_message
 from mailing.services.campaigns import queue_campaign
+from mailing.services.categories import TRANSACTIONAL_CATEGORY, category_label, validate_canonical_category
 from mailing.services.cmp_callbacks import emit_cmp_contact_event
 from mailing.services.contacts import (
     assign_tag,
@@ -854,6 +855,37 @@ def category_preference_payload(preference, tag):
     }
 
 
+def validate_optional_category(value):
+    """Return the canonical category for a request, or None when absent.
+
+    Without ``category`` the subscribe/unsubscribe endpoints keep their
+    historical behavior exactly.
+    """
+    if value in (None, ""):
+        return None
+    return validate_canonical_category(value)
+
+
+def apply_canonical_category_preference(contact, audience, client, category, *, enabled, reason):
+    """Enable or disable the canonical CategoryPreference tag for one scope.
+
+    The canonical category name is stored as the preference tag, so send-time
+    suppression keeps working through the same tag path.
+    """
+    preference, _ = CategoryPreference.objects.update_or_create(
+        contact=contact,
+        audience=audience,
+        client=client,
+        tag=category,
+        defaults={
+            "label": category_label(category),
+            "enabled": enabled,
+            "updated_reason": (reason or "")[:255],
+        },
+    )
+    return preference
+
+
 def preferences_suppression_payload(contact):
     if contact is None:
         return {
@@ -1211,9 +1243,20 @@ def erase_contact_for_client(data, authenticated_client):
 def subscribe_for_client(data, authenticated_client):
     scope = validate_contact_scope(data, authenticated_client)
     tags = validate_tags(data.get("tags"))
+    category = validate_optional_category(data.get("category"))
     contact, _ = upsert_contact(scope.email)
     subscribe_contact(contact, scope.audience, scope.client)
 
+    if category is not None:
+        apply_canonical_category_preference(
+            contact,
+            scope.audience,
+            scope.client,
+            category,
+            enabled=True,
+            reason="subscribe",
+        )
+
     for tag_name in tags:
         assign_tag(contact, scope.audience, tag_name)
 
@@ -1230,6 +1273,9 @@ def unsubscribe_for_client(data, authenticated_client):
     reason = data.get("reason", "")
     if reason is not None and not isinstance(reason, str):
         raise ApiValidationError({"reason": "must_be_string"})
+    category = validate_optional_category(data.get("category"))
+    if category == TRANSACTIONAL_CATEGORY:
+        raise ApiValidationError({"category": "transactional_cannot_be_disabled"})
 
     contact, _ = upsert_contact(scope.email)
     reason = reason or ""
@@ -1242,6 +1288,16 @@ def unsubscribe_for_client(data, authenticated_client):
     else:
         unsubscribe_contact(contact, scope.audience, scope.client, reason=reason)
 
+    if category is not None:
+        apply_canonical_category_preference(
+            contact,
+            scope.audience,
+            scope.client,
+            category,
+            enabled=False,
+            reason=reason or "unsubscribe",
+        )
+
     return contact_status_payload(contact, scope.audience, scope.client, requested_email=scope.email) | {
         "scope": scope_name
     }