[observability] Grafana dashboard JSON for llm-proxy (Prometheus datasource)

[mfittko]

Summary

Provide a ready-to-import Grafana dashboard JSON for llm-proxy metrics, assuming a Prometheus datasource.

Architectural Guidance

• Dashboard must not require proprietary exporters.
• Keep it aligned to the metrics we actually expose (see Prometheus endpoint issue).
• Avoid overly complex panels; focus on operational usefulness.

Suggested Panels

• Request rate, error rate
• Latency (if available), or proxy upstream duration if tracked
• Cache: hits/misses/bypass/store (if exposed)
• Event bus: publish/drop, stream lag/pending (if available)
• Uptime / restarts

Delivery

• Store dashboard JSON in-repo (suggested path): deploy/helm/llm-proxy/dashboards/llm-proxy.json or docs/observability/dashboards/.
• Optionally: Helm can create a ConfigMap with grafana_dashboard label (opt-in) for Grafana sidecar discovery.

Acceptance Criteria

• Dashboard imports cleanly.
• Queries match the exported metric names.
• Docs show how to use it.

Parent: #54

[mfittko]

Ready to Implement 🚀

All prerequisites are now complete:

• ✅ [observability] Prometheus metrics endpoint (text exposition) #209 - Prometheus endpoint with text exposition format
• ✅ [observability] Add resource usage metrics to Prometheus endpoint #225 - Resource usage metrics (CPU, memory, goroutines, GC)
• ✅ [deploy/helm] Prometheus scraping integration (ServiceMonitor/annotations) #210 - Helm chart integration for scraping

Available Metrics for Dashboard:

Application Metrics:

• llm_proxy_uptime_seconds
• llm_proxy_requests_total
• llm_proxy_errors_total
• llm_proxy_cache_hits_total
• llm_proxy_cache_misses_total
• llm_proxy_cache_bypass_total
• llm_proxy_cache_stores_total

Go Runtime Metrics:

• llm_proxy_goroutines
• llm_proxy_memory_heap_alloc_bytes
• llm_proxy_memory_total_alloc_bytes
• llm_proxy_gc_runs_total
• And more...

This is the final task to complete epic #54. Let's create an awesome dashboard! 📊