Skip to content

Add config.yaml custom model routing with quota-aware provider selection#3394

Merged
hlohaus merged 3 commits intomainfrom
copilot/add-config-yaml-custom-routing
Mar 20, 2026
Merged

Add config.yaml custom model routing with quota-aware provider selection#3394
hlohaus merged 3 commits intomainfrom
copilot/add-config-yaml-custom-routing

Conversation

Copy link
Contributor

Copilot AI commented Mar 19, 2026

Adds LiteLLM-style config.yaml support for defining named model routes that fan out to real providers with optional quota/error-count conditions, quota caching, and 429-triggered cache invalidation.

Core module: g4f/providers/config_provider.py

  • QuotaCache – TTL-based cache (5 min) for get_quota() results per provider; immediately invalidated on 429 responses
  • ErrorCounter – Rolling 1-hour window error counter per provider
  • evaluate_condition() – Safe recursive-descent parser supporting provider-specific quota fields via dot-notation (e.g. quota.balance > 0, quota.credits.remaining > 0 or error_count < 3); no eval
  • RouterConfig – Parses and holds routes from config.yaml
  • ConfigModelProviderAsyncGeneratorProvider that iterates configured providers, evaluates conditions, records errors, and invalidates quota cache on 429

Integration

  • g4f/cookies.pyread_cookie_files() now loads config.yaml from the cookies directory alongside .har/.json files
  • g4f/client/service.pyget_model_and_provider() checks RouterConfig before standard model resolution, making custom names transparent to callers

Condition variables

Each provider returns its own quota format from get_quota(). Conditions support:

Variable Description
quota Full quota dict from get_quota() — access any field with dot-notation
quota.balance PollinationsAI: {"balance": float}
quota.credits.remaining Yupp: {"credits": {"remaining": int, "total": int}}
balance Shorthand alias for quota.balance (PollinationsAI backward compat)
error_count Recent errors for this provider in the last hour — works universally

Missing quota keys resolve to 0.0.

Config format

Place config.yaml in your cookies dir (~/.config/g4f/cookies/ or ./har_and_cookies/):

models:
  - name: "my-gpt4"
    providers:
      - provider: "OpenaiAccount"
        model: "gpt-4o"
        condition: "balance > 0 or error_count < 3"   # PollinationsAI-style
      - provider: "PollinationsAI"   # unconditional fallback
        model: "openai-large"

  - name: "yupp-chat"
    providers:
      - provider: "Yupp"
        model: "gpt-4o"
        condition: "quota.credits.remaining > 0 or error_count < 3"  # Yupp-specific
      - provider: "PollinationsAI"
        model: "openai-large"

  - name: "llama-fast"
    providers:
      - provider: "Groq"
        model: "llama-3.3-70b"
        condition: "error_count < 3"   # universal
      - provider: "DeepInfra"
        model: "meta-llama/Llama-3.3-70B-Instruct"

Requires pyyaml (already in requirements.txt); gracefully skipped with a log warning if absent.

See docs/config-yaml-routing.md for full condition syntax reference and Python API.

Original prompt

This section details on the original issue you should resolve

<issue_title>add config.yaml for custom model routing</issue_title>
<issue_description>Add config.yaml support from cookies/config dir like in litelm. Support if condiotion with (if get_qutoa.balance >0 or error_count < 3. Cache qutoa result, refresh on 429. Add all wanted/needed functions and documentaion, tests</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

Co-authored-by: hlohaus <983577+hlohaus@users.noreply.github.com>
Copilot AI changed the title [WIP] Add config.yaml support for custom model routing Add config.yaml custom model routing with quota-aware provider selection Mar 19, 2026
Copilot AI requested a review from hlohaus March 19, 2026 14:32
@hlohaus
Copy link
Collaborator

hlohaus commented Mar 19, 2026

@copilot quota.balance is only for pollinations. All providers have his own qouta stats format

Co-authored-by: hlohaus <983577+hlohaus@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Mar 19, 2026

@copilot quota.balance is only for pollinations. All providers have his own qouta stats format

Fixed in 8fd8cd5. evaluate_condition now accepts the full quota dict from get_quota() and exposes it as quota.* dot-notation in conditions, since each provider returns its own schema:

  • PollinationsAI{"balance": float}quota.balance > 0 (or the balance shorthand)
  • Yupp{"credits": {"remaining": int, "total": int}}quota.credits.remaining > 0
  • PuterJS, GeminiCLI, GithubCopilot → their own formats, accessed via quota.<field>
  • Missing keys resolve to 0.0 instead of raising

balance is kept as a backward-compat alias for quota.balance (PollinationsAI), and error_count < N continues to work universally for any provider.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • api.deepinfra.com
    • Triggering command: /usr/bin/python python -m etc.unittest (dns block)
  • api.puter.com
    • Triggering command: /usr/bin/python python -m etc.unittest (dns block)
  • chat.z.ai
    • Triggering command: /usr/bin/python python -m etc.unittest (dns block)
  • chat3.eqing.tech
    • Triggering command: /usr/bin/python python -m etc.unittest (dns block)
  • g4f.space
    • Triggering command: /usr/bin/python python -m etc.unittest (dns block)
  • https://api.github.com/repos/xtekky/gpt4free/releases/latest
    • Triggering command: /usr/bin/python python -m etc.unittest (http block)
  • huggingface.co
    • Triggering command: /usr/bin/python python -m etc.unittest (dns block)
  • image.pollinations.ai
    • Triggering command: /usr/bin/python python -m etc.unittest (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@hlohaus hlohaus marked this pull request as ready for review March 20, 2026 09:48
@hlohaus hlohaus merged commit d20b992 into main Mar 20, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

add config.yaml for custom model routing

2 participants