Feat/add litellm provider

[RheagalFire]

Related Issues

• None (new integration)

Proposed Changes:

Adds a new litellm-haystack integration package with a LiteLLMChatGenerator component that routes to 100+ LLM providers via litellm.completion() and litellm.acompletion().

Haystack has 60+ integrations but no LiteLLM integration. Users who want a single component that works across providers (OpenAI, Anthropic, Google, AWS Bedrock, Azure, Cohere, Mistral, Groq, etc.) currently have
no option. LiteLLM is the most popular AI gateway (45k+ stars) and this integration lets users access any supported provider with one component and standard environment variables.

The component supports:

• Sync (run) and async (run_async) execution
• Streaming with callback support (including tool call deltas)
• Tool calling with full serialization/deserialization
• drop_params=True by default for cross-provider compatibility
• Serialization round-trip via to_dict / from_dict (including Secret, tools, callbacks)

Files added:

• integrations/litellm/ -- full integration package
  • src/.../litellm/chat/chat_generator.py -- LiteLLMChatGenerator component
  • tests/test_litellm_chat_generator.py -- 21 unit tests
  • pyproject.toml, README.md

How did you test it?

Unit tests (21 passed):

TestLiteLLMChatGeneratorInit::test_default_init PASSED
TestLiteLLMChatGeneratorInit::test_init_with_params PASSED
TestSerialization::test_to_dict PASSED                    
TestSerialization::test_from_dict PASSED                                                                                                                                                                           
TestRun::test_basic_completion PASSED   
TestRun::test_drop_params_always_set PASSED                                                                                                                                                                        
TestRun::test_api_key_forwarded PASSED     
TestRun::test_api_key_omitted_when_none PASSED                                                                                                                                                                     
TestRun::test_base_url_forwarded PASSED       
TestRun::test_generation_kwargs_merged PASSED                                                                                                                                                                      
TestRun::test_empty_messages_returns_empty PASSED
TestRun::test_usage_in_meta PASSED                                                                                                                                                                                 
TestRun::test_tool_calls_parsed PASSED
TestRun::test_streaming_content_accumulated PASSED                                                                                                                                                                 
TestRun::test_generation_kwargs_runtime_overrides_init PASSED
TestRun::test_none_content_in_response PASSED                
TestRun::test_tools_sent_to_litellm PASSED                                                                                                                                                                         
TestSerializationRoundTrip::test_round_trip_basic PASSED
TestSerializationRoundTrip::test_round_trip_with_secret PASSED                                                                                                                                                     
TestAsync::test_run_async_basic PASSED                                                                                                                                                                             
TestAsync::test_run_async_empty_messages PASSED
============================== 21 passed in 2.11s ==============================                                                                                                                                   

Live E2E (Anthropic via Azure AI Foundry):
=== Live completion ===
Reply: OK
Meta: {'model': 'claude-sonnet-4-6', 'finish_reason': 'stop',
'usage': {'prompt_tokens': 13, 'completion_tokens': 4, 'total_tokens': 17}}

=== Streaming ===
Streamed chunks successfully
Full reply: 1, 2, 3, 4, 5

=== System + user messages ===
Reply: Ahoy there, matey! What brings ye to these here waters?

Notes for the reviewer

• This follows the same namespace package pattern as other integrations (no __init__.py at namespace levels)
• litellm>=1.60.0,<2.0 is pinned as a required dependency of this package only
• The component uses litellm.completion() / litellm.acompletion() directly (not through the OpenAI SDK)
• drop_params=True silently drops provider-unsupported kwargs for cross-provider compatibility
• Provider API keys are read from standard environment variables automatically (e.g. ANTHROPIC_API_KEY, OPENAI_API_KEY)

Example usage:

from haystack_integrations.components.generators.litellm import LiteLLMChatGenerator
from haystack.dataclasses import ChatMessage                                                                                                                                                                       
 
# Uses ANTHROPIC_API_KEY from environment automatically                                                                                                                                                            
generator = LiteLLMChatGenerator(
    model="anthropic/claude-sonnet-4-20250514",                                                                                                                                                                    
    generation_kwargs={"max_tokens": 1024},
)                                                                                                                                                                                                                  
 
messages = [ChatMessage.from_user("What's Natural Language Processing?")]                                                                                                                                          
result = generator.run(messages=messages)
print(result["replies"][0].text)                                                                                                                                                                                   

Checklist

• I have read the https://github.com/deepset-ai/haystack-core-integrations/blob/main/CONTRIBUTING.md and the https://github.com/deepset-ai/haystack-core-integrations/blob/main/CODE_OF_CONDUCT.md
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the https://www.conventionalcommits.org/en/v1.0.0/ for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test:.

[CLAassistant]

All committers have signed the CLA.