Most AI works in the demo and breaks with real users. I build the unglamorous part that decides whether it survives production — retrieval quality, evaluation, and reliability.
raggate — a thin, CI-gated evaluation gate for RAG & LLM systems. Golden set + pass/warn/fail bands + LLM-judge (or heuristic) scorers, shipped as a GitHub Action. Fails your build when answer quality regresses.
pip install raggate · MIT · tested on Python 3.10–3.13
Upstream fixes merged into the open-source LLM stack — provider parameter handling, tool calling, and async streaming.
| Repository | What I changed | Status |
|---|---|---|
| BerriAI/litellm #29779 | Fixed Cohere v2 chat silently dropping max_completion_tokens, so token limits now apply on the default route |
Merged |
| BerriAI/litellm #29812 | Preserved the forced-function tool_choice name through the Responses-to-Chat transform, so forced tool calls are no longer lost |
Merged |
| deepset-ai/haystack-core-integrations #3410 | Fixed async streaming chunk indices in GoogleGenAIChatGenerator to start at 0, correcting off-by-one chunk ordering |
Merged |
Focus — RAG · LLM agents · evaluation / LLMOps · retrieval · prompt-injection defense Languages — Python, TypeScript LLM stack — OpenAI (incl. Responses API), LiteLLM, Haystack, pgvector, FastAPI, Azure
Based in India · I build production-grade AI systems and help teams put real eval gates around their RAG/LLM pipelines · reach me on LinkedIn or email.