Setting promptTokens yields empty transcription: decode loop aborts on EOT sampled during forced prefill #501

Summary

Setting DecodingOptions.promptTokens (for example a vocabulary glossary) makes large-v3 turbo transcription return an empty string. There is no error, just an empty segment. The same audio transcribes correctly without promptTokens.

Reproduced on WhisperKit v1.0.0 and current main (dcf3a00), macOS 15 on Apple Silicon, model openai_whisper-large-v3-v20240930_turbo.

Root cause

TextDecoder.decodeText force-feeds the prefill (<|startofprev|> + prompt + <|startoftranscript|><|lang|><|task|><|notimestamps|>) one token per step. The token sampled at each of those steps is a throwaway, because the next iteration overrides it with the forced prompt token. The completion check still honors it:

let isSegmentCompleted =
    sampleResult.completed ||   // true if the throwaway sample is EOT
    currentTokens.count >= Constants.maxTokenContext - 1 ||
    isFirstTokenLogProbTooLow

With a <|startofprev|> prompt present, large-v3 turbo reliably samples <|endoftext|> while the SOT sequence is being forced, so the loop breaks before a single content token is decoded.

Debug trace with prompt " Claude Code, Kokoro, Authe":

Forcing prompt tokenIndex: 10, token: 50258, text: <|startoftranscript|>
Predicted next tokenIndex: 11, token: 50360, text: <|transcribe|>
Forcing prompt tokenIndex: 11, token: 50259, text: <|en|>
Predicted next tokenIndex: 12, token: 50257, text: <|endoftext|>   (loop aborts here)
Completed window: <|startoftranscript|><|en|><|transcribe|><|notimestamps|><|endoftext|>

The remaining prefill tokens (<|transcribe|>, <|notimestamps|>) are never forced and the result is an empty segment. This reproduces with different prompt contents, with detectLanguage: true, and with an explicit language: "en".

Repro

let wk = try await WhisperKit(WhisperKitConfig(model: "openai_whisper-large-v3-v20240930_turbo"))
let prompt = wk.tokenizer!.encode(text: " Claude Code, Kokoro, Authe")
    .filter { $0 < wk.tokenizer!.specialTokens.specialTokenBegin }
let options = DecodingOptions(withoutTimestamps: true, promptTokens: prompt, chunkingStrategy: .vad)
let results = try await wk.transcribe(audioArray: anySpeechSamples, decodeOptions: options)
// results text is "". Without promptTokens the same audio transcribes correctly.

Fix

Gate the completion check on the prefill being fully consumed:

let isSegmentCompleted =
    (sampleResult.completed && !isPrefill) ||
    ...

A real EOT at or after the last prefill token still completes the segment as before. Fix is in #502.