-
-
Notifications
You must be signed in to change notification settings - Fork 6k
Open
Labels
Description
Description
get_standard_logging_object_payload() in litellm_logging.py runs on every LLM call, even when no callbacks (success_callback, failure_callback, etc.) are configured.
For most providers this is cheap (model info comes from a static JSON map), but for Ollama it triggers an extra HTTP call to /api/show to fetch model metadata (context window, function_calling support). This means every litellm.completion() call to Ollama makes 2 HTTP requests: one to the LLM and one for model info that nobody uses.
Proposal
Before assembling the logging payload, check if any callbacks are registered. If none, skip the payload assembly entirely.
Something like:
if not (litellm.success_callback or litellm.failure_callback or litellm._async_success_callback or ...):
return None # no one will consume this payloadBenefits
- Eliminates unnecessary work on every LLM call for SDK users without callbacks
- Removes an extra HTTP round-trip per Ollama call when no observability is configured
- Prevents errors/warnings in logs from failed model info fetches when the info isn't needed anyway
Related
Reactions are currently unavailable