Is your feature request related to a problem? Please describe.
I have a llm deployed locally and cannot use OpenAI.
Describe the solution you'd like
I would like to access the open source project: xinference. Xinference gives you the freedom to use any LLM you need. It is much more powerful than ollama! Address: https://github.com/xorbitsai/inference.
Describe alternatives you've considered
Ollama is also possible, but not recommended because it supports too few models
Additional context
For projects involving company knowledge bases, everyone hopes that the data will not be leaked, so it is necessary to use local LLM.
Thank you for your project, it's amazing, I hope you can consider this demand！！
Is your feature request related to a problem? Please describe.
I have a llm deployed locally and cannot use OpenAI.
Describe the solution you'd like
I would like to access the open source project: xinference. Xinference gives you the freedom to use any LLM you need. It is much more powerful than ollama! Address: https://github.com/xorbitsai/inference.
Describe alternatives you've considered
Ollama is also possible, but not recommended because it supports too few models
Additional context
For projects involving company knowledge bases, everyone hopes that the data will not be leaked, so it is necessary to use local LLM.
Thank you for your project, it's amazing, I hope you can consider this demand！！