generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 259
Open
Labels
needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.Indicates an issue or PR lacks a `triage/foo` label and requires one.
Description
Summary
Valid POST /v1/embeddings requests (OpenAI-compatible embeddings API) are treated as chat-completions or completions by the extension. Because the request body has model and input instead of messages or prompt, the extension returns “failed to extract request data” or “invalid chat-completions request: chat-completions request must have at least one message” when the body doesn’t match (e.g. embeddings with input instead of messages).
The extension does not recognize the embeddings API path or request shape, so it never accepts embeddings as valid.
Root cause
Request body handling — pkg/epp/util/request/body.go
- Path dispatch:
determineAPITypeFromPath(lines 57–73) only handles/v1/conversations,/v1/responses,/v1/chat/completions, and/v1/completions. There is no branch for/v1/embeddings, so the path falls through to the default (completionsAPI). - Body parsing:
ExtractRequestBodyhas no case for an embeddings-shaped body (model+input). Requests are therefore interpreted as completions (expectingprompt) or, when the path is normalized elsewhere, as chat-completions (expectingmessages), which produces the errors above.
Response handling — pkg/epp/handlers/response.go
- Usage extraction:
extractUsageByAPITypeonly has explicit handling forresponse,conversation,chat.completion,chat.completion.chunk, andtext_completion. Embeddings responses use top-level"object": "list"and the same usage fields (prompt_tokens,total_tokens,completion_tokens). There is no explicit embeddings/list type, so usage extraction falls back to the default branch and embeddings are not a first-class API type.
Expected behavior
- Request:
POST /v1/embeddingswith a body such as:(or{ "model": "text-embedding-3-small", "input": "The food was delicious!" }inputas an array of strings) is recognized as an embeddings request and accepted. - Response: Embeddings responses are explicitly recognized for usage and metrics instead of relying on the default branch.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
needs-triageIndicates an issue or PR lacks a `triage/foo` label and requires one.Indicates an issue or PR lacks a `triage/foo` label and requires one.