Skip to content

fix: normalize i16 to f32 in segmentation and flush trailing speech - #28

Open
gregoire22enpc wants to merge 2 commits into
thewh1teagle:mainfrom
gregoire22enpc:fix/segmentation-normalization
Open

fix: normalize i16 to f32 in segmentation and flush trailing speech#28
gregoire22enpc wants to merge 2 commits into
thewh1teagle:mainfrom
gregoire22enpc:fix/segmentation-normalization

Conversation

@gregoire22enpc

@gregoire22enpc gregoire22enpc commented Apr 5, 2026

Copy link
Copy Markdown

Problem

get_segments in segment.rs has two bugs:

1. Missing normalization in i16→f32 conversion (line 59)

// Current (broken):
let array = ndarray::Array1::from_iter(window.iter().map(|&x| x as f32));

// Fixed:
let array = ndarray::Array1::from_iter(window.iter().map(|&x| x as f32 / 32768.0));

The ONNX segmentation model was trained on input in [-1.0, 1.0], but x as f32 feeds it raw i16 values in [-32768, 32767]. The model's weights, biases, and normalization layers are calibrated for the expected range, so values 32768x larger produce unreliable frame classifications — in practice, all frames are classified as non-speech.

This was discovered while integrating pyannote-rs in silverstein/minutes, where minutes enroll consistently failed with "No speech detected in the recording" and minutes record produced transcripts with missing speaker labels.

2. Trailing speech segment dropped at end-of-audio

When speech extends to the end of the audio (i.e., is_speeching is still true after all windows are processed), the final segment is silently dropped because there is no flush at the end of the iterator. This fix emits the trailing segment.

Changes

  • segment.rs line 59: Divide by 32768.0 when converting i16 to f32
  • segment.rs end of iterator: Flush the last speech segment when is_speeching is still true after processing all windows

Verification

cargo check passes. The fix is a minimal, targeted change to the existing logic.

Two bugs in get_segments:

1. i16 samples are cast to f32 via `x as f32` without dividing by
   32768, feeding the ONNX segmentation model values in [-32768, 32767]
   when it expects [-1.0, 1.0]. This causes the model to misclassify
   all frames as non-speech for typical microphone input.

2. When speech extends to end-of-audio, the final segment is silently
   dropped because there is no flush when `is_speeching` is still true
   after all windows are processed.

Made-with: Cursor
gregoire22enpc added a commit to gregoire22enpc/minutes that referenced this pull request Apr 5, 2026
pyannote-rs's get_segments casts i16 to f32 without dividing by 32768,
feeding the ONNX segmentation model values in [-32768, 32767] when it
expects [-1.0, 1.0]. This causes the model to classify all frames as
non-speech, breaking enrollment and diarization entirely.

Switch to a patched fork (gregoire22enpc/pyannote-rs@ae9fbc1) that:
1. Normalizes i16→f32 with / 32768.0 in get_segments
2. Flushes trailing speech segments at end-of-audio

Upstream PR: thewh1teagle/pyannote-rs#28
Once merged, we'll switch back to the crates.io release.

Made-with: Cursor
gregoire22enpc added a commit to gregoire22enpc/minutes that referenced this pull request Apr 5, 2026
pyannote-rs's get_segments casts i16 to f32 without dividing by 32768,
feeding the ONNX segmentation model values in [-32768, 32767] when it
expects [-1.0, 1.0]. This causes the model to classify all frames as
non-speech, breaking enrollment and diarization entirely.

Switch to a patched fork (gregoire22enpc/pyannote-rs@ae9fbc1) that:
1. Normalizes i16→f32 with / 32768.0 in get_segments
2. Flushes trailing speech segments at end-of-audio

Upstream PR: thewh1teagle/pyannote-rs#28
Once merged, we'll switch back to the crates.io release.

Made-with: Cursor
The trailing speech flush ran unconditionally after every window,
prematurely splitting segments that span a 10-second window boundary.
Gate it with `else` so it only fires when start_iter is exhausted.

Extract duplicated segment-creation logic into make_segment() to
eliminate copy-paste between the speech→silence handler and the
end-of-audio flush.

Made-with: Cursor
@gregoire22enpc

Copy link
Copy Markdown
Author

@thewh1teagle, happy to get your thoughts on this :)

silverstein pushed a commit to silverstein/minutes that referenced this pull request Apr 6, 2026
Points pyannote-rs to a patched fork that fixes the i16-to-f32
normalization bug (missing / 32768.0) and adds trailing speech flush.

This unblocks minutes enroll which was failing with "No speech detected."

Upstream PR: thewh1teagle/pyannote-rs#28
Tracking issue: #79
@silverstein

Copy link
Copy Markdown

Hi @thewh1teagle, we're using pyannote-rs in Minutes and this fix unblocks voice enrollment for our users. We're currently pinned to a fork with this patch. Happy to help test or co-maintain if that would be useful. Any timeline for a release?

silverstein pushed a commit to silverstein/minutes that referenced this pull request Apr 9, 2026
Points pyannote-rs to a patched fork that fixes the i16-to-f32
normalization bug (missing / 32768.0) and adds trailing speech flush.

This unblocks minutes enroll which was failing with "No speech detected."

Upstream PR: thewh1teagle/pyannote-rs#28
Tracking issue: #79
akinsella added a commit to knsl-io/pyannote-rs that referenced this pull request May 23, 2026
…gmentation

Squashes upstream PR thewh1teagle#28 (thewh1teagle#28)
by gregoire22enpc. The segmentation ONNX expects samples in [-1.0, 1.0]
but PR thewh1teagle#22 (and the pre-22 code) casts i16 directly to f32, feeding the
model values in [-32768, 32767]. The model classifies everything as
non-speech and the iterator emits zero segments. A '/ 32768.0' division
restores correct scaling.

Co-Authored-By: gregoire22enpc <noreply@github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants