OTEL Support

[dbrun3]

ref:
https://github.com/OutThereLabs/actix-web-opentelemetry

Could keep the setup the same on the django end:
https://github.com/open-telemetry/opentelemetry-python/blob/main/docs/examples/django/README.rst

thoughts?

[FarhanAliRaza]

Opentelemetry is good to have. Not a real need for now i think. There are some more important problems for now. Like memory issue for long running servers.
If more users of the framework want it, we can add it later.

[jameswinegar]

OpenTelemetry Support — Considerations

Exploratory reference, not a spec. Tracking issue:
dj-bolt/django-bolt#242.
Living doc, expected to drift — code references are ~2026-06 pointers and will
stale. Treat the reasoning as durable, the specifics as illustrative.

0. The headline: django-bolt sits on top of Django, not within it

Django-bolt uses Django as a library (ORM, settings, models, auth, admin) but
not its request machinery — no django.core.handlers, no MIDDLEWARE chain,
no URL resolver, no WSGI/ASGI request cycle. Requests reach Python already routed,
authed, and parsed by Rust/Actix. So most "OTel + Django" guidance — including the
one linked in the issue — assumes you're inside Django's request path and does
not apply:

Django-ecosystem OTel approach	Works?	Why
DjangoInstrumentor().instrument() (the linked guide)	❌	Patches Django's request handling, which we never invoke → zero spans.
MIDDLEWARE / WSGI / ASGI-app instrumentation	❌	We don't run Django's request chain (except admin mounts).
Django request/response signals	⚠️	Only if BOLT_EMIT_SIGNALS is on — unreliable hook.
Library instrumentors: DB (psycopg/mysqlclient), cache (redis), HTTP clients (requests/httpx)	✅	Hook the libraries Django uses, independent of who drives the request. Keep these.
Django settings as a config source	✅	That's us reading settings, not Django wiring OTel.

Consequence: the request/server span is django-bolt's own responsibility
(must originate in Rust, before Python runs). Django's ecosystem gives us the
sub-spans (DB/cache/outbound) for free — but not the request span, and not a
turnkey setup. "Same setup on the Django end" is only half-true.

The same mismatch exists on the Rust side, and it's why this isn't a wiring
exercise. The issue references actix-web-opentelemetry, which assumes it owns
the whole request → span → export pipeline in Rust. But because export is best
kept out of the wheel and moved to Python (§4), and the handler span lives in
Python, that crate collapses to — at most — inbound context extraction and timing.
So neither ecosystem's turnkey tool fits: not Django's (wrong request path),
not actix's (wrong home for spans/export). django-bolt is a hybrid and has to
assemble the pieces itself.

1. Two-runtime problem → main fork

A complete trace needs a root span in Rust (accurate timing, before the GIL)
and handler/DB/client spans in Python, nested under one sampling decision
that crosses the PyO3 boundary. Where spans live is the key decision:

Option	Trade-off
Rust-only (Python returns timings)	Loses all Django library instrumentation.
Python-only (Rust passes timings up)	Server timing polluted by GIL bounce; metrics through Python is expensive.
Dual-SDK, shared trace_id (leaning this way)	Native context each side; library instrumentation just works; cheap Rust metrics. Cost: two exporters, consistent Resource.

2. The bridge (the crux — spike this first)

• Extract inbound context in Rust via the configured propagator; start the server span.
• Decide sampling once, in Rust — single source of truth.
• Carry span context (trace_id/span_id/flags/tracestate) to Python without a
  per-request allocation (lazy fields, like the existing OnceLock pattern).
• Rebuild it Python-side as a remote parent; start the handler span under it.
• Python must use a ParentBased sampler so it respects Rust's decision —
  else partial traces. Easiest thing to get subtly wrong.

Hazards to verify, not assume: sync_to_thread handlers (contextvars don't cross
thread pools → orphaned DB spans); contextvars surviving the pyo3-async-runtimes
hand-off; providers init per worker after fork (OTLP channels don't survive
fork()); force_flush on lifespan/SIGTERM.

3. Coverage checklist

Server span (Rust, http.route = template not raw path) · auth/guards/rate-limit
(no secrets) · Python handler span · DB/cache/outbound via library instrumentors ·
streaming (span ends at stream completion) · WebSockets & admin mount (separate
paths) · exceptions (record_exception) · RED metrics in Rust (cheap, useful
even when unsampled — highest value/cost) · log↔trace correlation.

4. Distribution: how users opt in without bloating the wheel

CI ships prebuilt abi3 wheels to PyPI — most users never compile. So a Cargo
feature is a maintainer/CI knob, not a user one: one shared wheel means
feature-on bloats everyone, feature-off locks everyone out. The bloat is
specifically the gRPC exporter (opentelemetry-otlp → tonic/prost/h2). Options:

• Recommended — keep opentelemetry-otlp out of the Rust binary; gate via a
  Python extra. Rust contributes only trace-context parse/format + accurate
  start/end timestamps (stashed on the request); Python materializes the server
  span (explicit start/end times) and owns the single OTLP exporter.
  pip install django-bolt[otel] pulls the Python SDK/exporter/instrumentors —
  no recompile, no wheel variant, zero bloat for non-users. Mirrors the existing
  [mcp] companion-package pattern. (Consolidates export into one Python pipeline.)
• If Rust-side spans are wanted: light opentelemetry/_sdk core only (never
  -otlp), export via Python; or if a Rust exporter is non-negotiable, OTLP/HTTP
  (reuse the hyper stack actix already pulls), not gRPC.
• Escape hatch for --no-binary/sdist installs: an otel Cargo feature,
  default-off, via --config-settings=build-args="--features otel" (needs Rust
  toolchain; advanced-only).
• Discouraged: a separate django-bolt-otel wheel variant — doubles CI + binary.

Trade-off: Python export gives up the cheapest Rust-side RED metrics. Mitigate
with a tiny Rust metrics core (no -otlp needed), a Prometheus /metrics crate,
or Python-recorded metrics for v1.

5. Performance (hard requirement: no regression off, bounded on)

Compile-eliminated disabled path (extend the existing
handle_request<const ACCESS_LOG> monomorphization) · unsampled requests skip
span build + Python span · registration-time pre-compile · async batched export,
bounded queues · benchmark as a merge gate.

6. Pluggability (it's a library — deployments should change config, not code)

Honor OTEL_* env and a BOLT_OTEL settings dict (bare VMs have no env
injection) · configurable propagator both sides, W3C default (don't hardcode
traceparent or b3/xray meshes break) · pluggable exporter/sampler/resource ·
bring-your-own global provider (don't fight existing setup) · absent/unreachable
endpoint must degrade gracefully (never block startup/requests, never crash).

7. Security & trust

Span data hygiene (PII). Spans leak sensitive data by default — query params,
headers, cookies, user ids, request/response bodies. Default attributes must stay
conservative: http.route (template) yes; raw path/query/body no. Capturing
headers, query, or body must be opt-in and run through the same obfuscation
policy the logging middleware already has (obfuscate_headers/obfuscate_cookies

• field allowlist) — one policy, not two. Never emit auth secrets/tokens as
  attributes.

Trust boundary of inbound context. Honoring a client-supplied traceparent
with sampled=1 lets an untrusted caller force sampling — a trace-volume/cost
(DoS) vector on public ingress — and it interacts with the ParentBased sampler
(§2), which by spec respects the incoming flag. Stance: follow the OTel spec
default (extract + ParentBased), but treat whether inbound context is trusted
at this edge as explicit, user-owned configuration with validation — not a
silent default. A trusted internal mesh hop and a public edge want opposite
answers; the library should not pick for the user.

8. Deployment vectors to validate

Three orthogonal cases cover the axes that matter (a 4th like GKE/ECS re-tests
covered ground):

Axis	Cloud Run	EKS / K8s	Bare VM
CPU/lifecycle	throttled, scale-to-zero	continuous	continuous
Collector	sidecar	DaemonSet/gateway/Operator	maybe none
Config source	injected env	injected env	settings/code
Endpoint	reachable	reachable	maybe air-gapped
Stresses	export under CPU throttling; flush in ~10s SIGTERM grace	propagator/topology/BYO-provider	default-off no-op; graceful failure on unreachable endpoint

9. Sequencing & open questions

Likely order: (1) spike the bridge, (2) metrics-first (no cross-boundary work,
ships alone), (3) core tracing + perf gate + pluggability, (4) long tail
(streaming/WS/admin, log export).

Open questions:

• Default export protocol for the Python side — OTLP/HTTP vs Prometheus?
• Metrics before tracing?
• Where RED metrics live given export is in Python — tiny Rust core vs Prometheus
  endpoint vs Python-recorded (see §4 trade-off)?
• v1 scope — HTTP only, or also WebSockets + admin mount?
• traceresponse (Level 2, low adoption) as an off-by-default opt-in seam?
• Which OTel HTTP semantic-convention version to pin? They churn (e.g.
  http.method → http.request.method), so target a current stable semconv and
  revisit on upgrade.