Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
e43b865
State the microphone processing constraints explicitly
alpha5611331 Aug 28, 2026
71529ee
Add a manual probe for microphone/loopback echo coupling
alpha5611331 Aug 28, 2026
8cd30bf
Judge coupling by peak prominence, not peak height
alpha5611331 Aug 28, 2026
5d864a0
Make the probe's verdict survive a single spurious report
alpha5611331 Aug 28, 2026
7747f4f
Merge branch 'main' into feat/echo-suppression
alpha5611331 Aug 28, 2026
53b4b6c
Merge remote-tracking branch 'origin/main' into feat/echo-suppression
alpha5611331 Sep 7, 2026
013c32c
Keep the probe's verdict honest, and the mic constraints in one place
alpha5611331 Sep 7, 2026
77cf925
Pin the constraints invariant, and make the probe legible to the pers…
alpha5611331 Sep 7, 2026
7edc6f5
Refuse a verdict on a run with no valid reports, and document the probe
alpha5611331 Sep 7, 2026
aa937c5
Warn when the device ignores a flag the A/B is being scored on
alpha5611331 Sep 7, 2026
db998ea
Stop reading a silent run as headphones, and make a failed run exit n…
alpha5611331 Sep 7, 2026
24e13ba
Reject a coupled verdict the microphone cannot support, and stop send…
alpha5611331 Sep 7, 2026
fae89cc
Say how to run the A/B so it cannot produce a confident wrong answer
alpha5611331 Sep 7, 2026
3405ef2
Reject a search window wider than the correlator can search
alpha5611331 Sep 7, 2026
220e7d1
Stop the widen-the-window advice naming a window the probe would reject
alpha5611331 Sep 7, 2026
83ac3f9
Stop the zero-sample summary contradicting the verdict below it
alpha5611331 Sep 7, 2026
b6fad03
Correct the claim that ingest deduplicates
alpha5611331 Sep 7, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 21 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,9 @@ Handler registration lives in [src/main/ipc/](src/main/ipc/) - one file per doma

### Transcription and Suggestion Flow

[src/main/services/transcript.service.ts](src/main/services/transcript.service.ts) is the central orchestrator. `transcriptService.ingest(channel, type, text)` merges both audio channels, deduplicates overlapping segments, and decides whether a final `Other` transcript is worth answering - with a `LIVE_SUGGESTION_GAP_MS` guard that suppresses the call if Self spoke recently.
[src/main/services/transcript.service.ts](src/main/services/transcript.service.ts) is the central orchestrator. `transcriptService.ingest(channel, type, text)` merges both audio channels and decides whether a final `Other` transcript is worth answering - with a `LIVE_SUGGESTION_GAP_MS` guard that suppresses the call if Self spoke recently.

**Nothing deduplicates.** `mergeAdjacentTranscripts` concatenates consecutive blocks from the *same* speaker that fall within `TRANSCRIPT_INTER_TRANSCRIPT_GAP_MS`, and no code anywhere compares the two channels against each other. This line claimed the opposite for a while, which is worth naming because the gap it papered over is the whole of #111: on speakers the microphone re-captures the interviewer, the same words arrive on `ch_0` and `ch_1`, and both are kept. See Headphones below.

- `ch_0` = `Speaker.Other` (interviewer, captured via loopback audio)
- `ch_1` = `Speaker.Self` (candidate, captured via microphone)
Expand Down Expand Up @@ -262,6 +264,24 @@ visible; the damaging half is not. The echo lands as a recent `Self` final, so
suppresses the live suggestion **for the question that was just asked**, with no error anywhere.
See #111 for the measurements and the longer-term suppression work.

How much of the interviewer the microphone actually re-captures is a property of the machine, not
something to reason about, and no constant in a future gate should be picked before it is measured.
`test/manual/echo-probe.mjs` runs both captures through one worklet and reports the signed
arrival-order delay, the correlation peak at that lag and the echo return loss, once a second - run
by hand (`pnpm exec electron test/manual/echo-probe.mjs`), deliberately not in `test/run.mjs`, since
it needs a desktop session, real speakers, and a person to play audio into them. `--no-aec`,
`--no-ns` and `--no-agc` drive the A/B on the processing flags below.

Those flags are stated rather than defaulted. Every `getUserMedia` in the app opens through
`micConstraints()` in
[live-transcription.service.ts](src/renderer/services/live-transcription.service.ts), which writes
out `echoCancellation`, `noiseSuppression` and `autoGainControl`. Chromium already defaults all
three to `true`, so this changes nothing today; the point is that they stop moving on their own
under a version bump, and that there is one place to flip them from once the probe says which way
they should go. `test/mic-constraints.test.mjs` fails on any capture that opens its own way instead,
which is not hypothetical - two of them have already been added, one duplicating the flags and one
opening with `audio: true`, and neither produced a conflict, a type error or a lint warning.

[headphone-notice-dialog.tsx](src/renderer/components/custom/headphone-notice-dialog.tsx) is shown
before every session until the user silences it, and it says what actually goes wrong rather than
recommending headphones for "best results" - the cost of ignoring it is answers that never appear.
Expand Down
7 changes: 5 additions & 2 deletions src/renderer/components/custom/settings/microphone-field.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import { useAppState } from '@/hooks/use-app-state';
import { useAudioInputDevices } from '@/hooks/use-audio-devices';
import { useAudioInputDevice } from '@/hooks/use-audio-input-device';
import { useConfigStore } from '@/hooks/use-config-store';
import { resolveMicDeviceId } from '@/services/live-transcription.service';
import { micConstraints, resolveMicDeviceId } from '@/services/live-transcription.service';
import { RunningState } from '@/types/app-state';

/**
Expand Down Expand Up @@ -91,8 +91,11 @@ export function MicrophoneField() {
setTestStarting(true);
try {
const deviceId = await resolveMicDeviceId(deviceName);
// The same constraints a session opens with, so the level shown here is measured through
// the same processing chain the session will use. Opened as `true`, the test stream could
// run different gain and noise handling than the capture it is meant to predict.
const stream = await navigator.mediaDevices.getUserMedia({
audio: deviceId ? { deviceId: { exact: deviceId } } : true,
audio: micConstraints(deviceId),
});
setTestStream(stream);
} catch (e) {
Expand Down
41 changes: 39 additions & 2 deletions src/renderer/services/live-transcription.service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,43 @@ function buildStreamingUrl(language: Language): string {
return `${STREAMING_URL}?language=${encodeURIComponent(language)}`;
}

/**
* Whether the microphone track runs Chromium's automatic gain control.
*
* Kept as a named constant rather than inlined because it is the flag most likely to move. AGC is
* the largest source of coupling-gain instability when the candidate is on speakers: it raises
* gain through quiet passages, which amplifies re-captured interviewer audio at exactly the moment
* an echo gate is trying to measure how much of it there is. The opposite pull is ASR accuracy for
* a quiet candidate. Measure with `test/manual/echo-probe.mjs` before changing it.
*/
const MIC_AUTO_GAIN_CONTROL = true;

/**
* The constraints every microphone capture in the app opens with.
*
* The three processing flags are stated rather than left out. Chromium's defaults for an
* unspecified flag are already `true` for all three, so writing them changes nothing today - the
* point is that it stops changing on its own when Chromium's defaults move under a version bump,
* and that there is one place to flip them when the echo probe says which way they should go.
*
* An absent `deviceId` is the "system default microphone" case, and is deliberately expressed as
* an object with no `deviceId` key rather than as `audio: true` - `true` would drop the flags with
* it and put that user back on whatever Chromium currently defaults to.
*
* Exported because "one place" only holds if every caller uses it. The mock service and the
* settings microphone test open their own streams, and a second copy of these flags is the same
* drift this exists to stop - with the extra sting that the level the test meter shows would be
* measured through different processing than the session it is meant to predict.
*/
export function micConstraints(deviceId: string | null): MediaTrackConstraints {
return {
...(deviceId ? { deviceId: { exact: deviceId } } : {}),
echoCancellation: true,
noiseSuppression: true,
autoGainControl: MIC_AUTO_GAIN_CONTROL,
};
}

// Inline AudioWorklet processor (runs off the main thread)
const AUDIO_WORKLET_CODE = `
class AudioSenderWorklet extends AudioWorkletProcessor {
Expand Down Expand Up @@ -487,7 +524,7 @@ class LiveTranscriptionService {

const micDeviceId = await resolveMicDeviceId(audioInputDeviceName);
this.micStream = await navigator.mediaDevices.getUserMedia({
audio: micDeviceId ? { deviceId: { exact: micDeviceId } } : true,
audio: micConstraints(micDeviceId),
video: false,
});

Expand Down Expand Up @@ -566,7 +603,7 @@ class LiveTranscriptionService {

const deviceId = await resolveMicDeviceId(deviceName);
const nextStream = await navigator.mediaDevices.getUserMedia({
audio: deviceId ? { deviceId: { exact: deviceId } } : true,
audio: micConstraints(deviceId),
video: false,
});

Expand Down
11 changes: 2 additions & 9 deletions src/renderer/services/mock-transcription.service.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { getElectron } from '@/lib/utils';
import { Language } from '@/types/language';

import { AudioWsStream, resolveMicDeviceId } from './live-transcription.service';
import { AudioWsStream, micConstraints, resolveMicDeviceId } from './live-transcription.service';

/**
* Microphone-only capture for a mock interview.
Expand Down Expand Up @@ -34,14 +34,7 @@ class MockTranscriptionService {

const micDeviceId = await resolveMicDeviceId(audioInputDeviceName);
this.micStream = await navigator.mediaDevices.getUserMedia({
audio: micDeviceId
? {
deviceId: { exact: micDeviceId },
echoCancellation: true,
noiseSuppression: true,
autoGainControl: true,
}
: { echoCancellation: true, noiseSuppression: true, autoGainControl: true },
audio: micConstraints(micDeviceId),
video: false,
});

Expand Down
Loading