R1.1 Relay production environment

[alexeygrigorev]

Implements the code half of R1.1 (Relay production environment) from the community-base unification plan. Refs #13. The issue stays open for its owner-gated live checks.

What changed

1. scripts/deploy_relay_sandbox.sh takes --environment sandbox|production (default sandbox, plus --help). The sandbox path is behaviour-identical: same runtime env bootstrap, same containers, same rendered docker/git/curl commands. The production path changes only what the main/relay host contract requires: no PostgreSQL container and no Caddy (RDS and the shared ALB instead), web container bound to 0.0.0.0:8000, no inbound-mail ingress (production is outbound only), explicit --memory on every container (2 GiB host), log group /relay/production/host, SECRET_KEY and DATABASE_URL fetched from Secrets Manager via the ARNs in /etc/relay/infrastructure.env and never printed, and no public health curl (private host; the release gate is the local readiness check, the per-container checks and the in-host system.echo smoke test). ALLOWED_HOSTS, CSRF_TRUSTED_ORIGINS, DEFAULT_FROM_EMAIL, AWS_SES_REGION and AWS_SES_CONFIGURATION_SET are never rewritten by the deploy; RELAY_SES_MAX_SEND_RATE is not set in production, so the application default (10/s) applies.
2. .github/workflows/deploy-prod.yml: workflow_dispatch only, never on push. test job (ruff, makemigrations check, Django check, pytest) gates the deploy job, which runs under the production GitHub environment (owner configures it with a required reviewer), only from main, and sends the same SSM deploy command as sandbox with --environment production. Needs the repository variable RELAY_PRODUCTION_INSTANCE_ID; RELAY_PRODUCTION_DEPLOY_ROLE_ARN defaults to the relay-production-github-deploy role from the main/relay output.
3. scripts/provision_production_tenant.sh: provisions organisation datatalksclub, audience dtc, client dtc-website, sender hello=DataTalks.Club <hello@datatalks.club> (final sender address to be confirmed by the owner), and one API key named production-deployment. The key is generated on the host into the 0600 runtime env file, following the sandbox RELAY_BOOTSTRAP_API_KEY mechanism; nothing prints or embeds key material (output shows only the key name and public prefix). The owner copies the key into the DTC website deployment secrets. --help and --dry-run work without credentials. The production deploy invokes the script (it is idempotent), so tenant state converges on every deploy and the first deploy's smoke test passes; it can also be run standalone on the host.
4. docs/production.md: environment table, how a deploy works, tenant provisioning, on-call checks, rollback. Explicit that production is not deployed yet. README points at it.

Terraform note

No aws-infra change accompanies this PR: the plan's Terraform half is already satisfied by the merged main/relay root in DataTalksClub/aws-infra. Its documented deviations from the plan issue text (no Elastic IP: private host behind the shared ALB; one SES-webhook ingress queue: inbound mail is out of scope; the datatalks.club SES identity, DKIM, SPF and DMARC owned by main/common and main/website-static) are reported to the plan owner as a plan fix in docs/production.md; nothing is applied here.

One deployment obligation remains open on the owner side and is listed in the docs: main/relay schedules /usr/local/sbin/relay-prune-database daily at 03:30 UTC, but the application does not ship the underlying retention command yet, so the script cannot be installed by this deploy. The Secrets Manager value containers (database-url, django-secret-key) must also be populated out of band before the first deploy.

Verification

uv run pytest -> pass:

562 passed in 160.16s (0:02:40)

uv run ruff check . -> pass:

All checks passed!

Workflow YAML parses (yaml.safe_load on both):

.github/workflows/deploy-prod.yml OK triggers: ['workflow_dispatch'] jobs: ['test', 'deploy']
.github/workflows/deploy-sandbox.yml OK triggers: ['workflow_dispatch', 'push'] jobs: ['test', 'deploy']

bash -n on both scripts -> pass. --help and --dry-run work without credentials; the dry run prints the plan and never touches files.

Sandbox-path equivalence: the old script (924ac93), the new script with default arguments, and the new script with explicit --environment sandbox were each executed under command stubs (docker, git, curl, openssl, aws) inside a user+mount namespace with tmpfs /etc, /opt, /var/lib. The rendered command logs are byte-identical:

docker: old == new-default == new-explicit (41 lines)
git:    old == new-default == new-explicit (7 lines)
curl:   old == new-default == new-explicit (3 lines)
stdout identical
stderr identical

The same harness rendering --environment production confirms the expected differences: no postgres/caddy/inbound containers or pulls, --memory on all six app containers, gunicorn --bind 0.0.0.0:8000, awslogs-group=/relay/production/host, two secretsmanager get-secret-value calls with no secret value in any output, the tenant provisioning commands, no public curl, and the Relay production deployed: <sha> final line.

Not run here, needs: owner approved and applied the Terraform root and the first production deploy

• Canary email through the production path.
• datatalks.club SES identity verified in the AWS console.
• curl system.echo against the production deployment.
• GET /internal/ops/status from the production host.

The plan issue's Done-when boxes are left unchecked for these owner-gated checks.