Feature request
Currently, logit_bias is labeled as unused https://github.com/huggingface/text-generation-inference/blob/main/router/src/lib.rs
Also, the documentation states that a JSON is needed with a mapping from token_ids to a number between -100 and 100. But this is misaligned with the code's typing, that asks for a vector of floats.
Motivation
Logit_bias is a very important parameter, the documentation in InferenceClient from huggingface_hub states that it can be used, but it does not work.
Also see:
huggingface/huggingface_hub#2720
Your contribution
I can definitely help building a PR but I will need details on what type of solution is expected.
Feature request
Currently, logit_bias is labeled as unused https://github.com/huggingface/text-generation-inference/blob/main/router/src/lib.rs
Also, the documentation states that a JSON is needed with a mapping from token_ids to a number between -100 and 100. But this is misaligned with the code's typing, that asks for a vector of floats.
Motivation
Logit_bias is a very important parameter, the documentation in InferenceClient from huggingface_hub states that it can be used, but it does not work.
Also see:
huggingface/huggingface_hub#2720
Your contribution
I can definitely help building a PR but I will need details on what type of solution is expected.