fix(vertexai,gateway): eliminate credential race conditions in multi-user deployments

[Xibao-Lv]

Description

This PR addresses a critical thread-safety issue in the Gateway's VertexAI provider implementation that caused credential mixing in multi-user scenarios. The original implementation used global environment variables
(GOOGLE_APPLICATION_CREDENTIALS, GOOGLE_CLOUD_PROJECT, GOOGLE_CLOUD_LOCATION) to pass credentials, creating race conditions when multiple users with different GCP configurations made concurrent requests.

The refactor eliminates environment variable dependency entirely by:

• Passing credentials, project, and location directly to the VertexAI provider constructor
• Building isolated credential objects per provider instance
• Removing the obsolete vertex_auth.py module and temporary file management
• Simplifying Gateway configuration handling

PR Type

• 🐛 Bug Fix (fixes credential race conditions)
• 💅 Refactor (architectural changes for thread safety)

Relevant issues

Fixes #1016 - VertexAI credential race conditions in multi-user Gateway deployments

Checklist

• I understand the code I am submitting.
• I have added unit tests that prove my fix/feature works
• I have run this code locally and verified it fixes the issue.
• New and existing tests pass locally
• Documentation was updated where necessary
• I have read and followed the contribution guidelines
• AI Usage:
  • No AI was used.
  • AI was used for drafting/refactoring.
  • This is fully AI-generated.

AI Usage Information

• AI Model used: Claude Opus 4.6 (via Claude Code)
• AI Developer Tool used: Claude Code
• Any other info you'd like to share: AI assisted with code analysis, diff review, and documentation drafting, but architectural decisions and implementation were human-directed.

When answering questions by the reviewer, please respond yourself, do not copy/paste the reviewer comments into an AI system and paste back its answer. We want to discuss with you, not your AI :)

• I am an AI Agent filling out this form (check box if true)

Changes Made

Core Changes

1. src/any_llm/providers/vertexai/vertexai.py (_init_client method):
   - Modified to accept credentials, project, location as direct parameters
   - Builds isolated credential objects via _build_credentials()
   - Eliminates environment variable dependency
2. src/any_llm/providers/gemini/base.py (new method):
   - Added _build_credentials() to handle credential parsing from file paths, JSON strings, or dictionaries
   - Returns isolated Credentials objects with required GCP scopes
3. src/any_llm/gateway/api/routes/chat.py (get_provider_kwargs function):
   - Simplified to return configuration directly without side effects
   - Removed VertexAI-specific environment variable setup logic

Code Removal

4. src/any_llm/gateway/auth/vertex_auth.py (deleted):
   - Removed setup_vertex_environment() function and temporary file management
   - Eliminated global state and cleanup mechanisms
5. tests/gateway/test_vertex_credential_cleanup.py (deleted):
   - Tests for removed functionality are no longer relevant

Configuration Updates

6. docker/config.example.yml:
   - Removed reference to deleted vertex_auth.py module
   - Added clarification that credentials accepts file paths, JSON strings, or dictionaries
7. docs/src/content/docs/gateway/configuration.md:
   - Added clarifying comments for VertexAI configuration
   - Documented that credentials accepts file paths, JSON strings, or dictionaries
   - Updated to reflect new architectural approach
   - Note: Configuration structure remains credentials, project, location at top level (not nested under client_args)

Technical Benefits

• Thread Safety: Each provider instance uses isolated credentials, preventing race conditions in multi-user scenarios
• Security: Eliminates credential/project leakage between concurrent users
• Simplification: Removes complex temporary file management and cleanup logic
• Performance: Reduces overhead from environment variable manipulation and file I/O

Configuration Impact

Users must update their configuration:

#### before
  vertexai:
    credentials: "/path/to/service_account.json"  # File path, JSON string, or dict
    project: "your-gcp-project-id"
    location: "us-central1"


### after 
  vertexai:
    client_args:
      credentials: "/path/to/service_account.json"  # File path, JSON string, or dict
      project: "your-gcp-project-id"
      location: "us-central1"

Files Changed: 7 files, 79 insertions(+), 200 deletions(-)
Commit History: 4 commits focusing on refactoring, cleanup, and documentation

Key Fix: Eliminates environment variable dependency that caused race conditions in multi-user Gateway deployments, ensuring each request uses isolated credentials specific to that user/configuration.

[tbille]

Hi @Xibao-Lv ,
Thank you very much for this contribution.

we are currently working on migrating the code of the gateway on a standalone repository. Would you mind reopening this PR on the new repository: https://github.com/mozilla-ai/gateway

[Xibao-Lv]

Hi @Xibao-Lv , Thank you very much for this contribution.

we are currently working on migrating the code of the gateway on a standalone repository. Would you mind reopening this PR on the new repository: https://github.com/mozilla-ai/gateway

ok, i will reopen it later.

[tbille]

Thanks @Xibao-Lv , In the meantime I will close this PR