Skip to content

feat: 增加 Owner 声纹录入与本地 Profile - #2459

Draft
MomiJiSan wants to merge 3 commits into
Project-N-E-K-O:codex/voice-identity-foundation-20260723from
MomiJiSan:codex/voice-owner-enrollment
Draft

feat: 增加 Owner 声纹录入与本地 Profile#2459
MomiJiSan wants to merge 3 commits into
Project-N-E-K-O:codex/voice-identity-foundation-20260723from
MomiJiSan:codex/voice-owner-enrollment

Conversation

@MomiJiSan

Copy link
Copy Markdown
Contributor

概要

这是 #2450 之上的第一个新增堆栈 PR,只实现 Owner 声纹录入、本地加密 Profile 与设置页面;测试版软拦截将在后续堆栈 PR 中实现。

包含 3 个逻辑 commit:

  1. 版本化 AES-GCM 本地 ProfileStore 与操作系统安全存储密钥。
  2. 3 段固定文案 + 2 次自由说话验证的录入 API、状态机和独占录入 lease。
  3. /voice_identity 设置页面、Web/Electron 录音流程及 8 个 locale。

边界

  • 不修改 Provider worker、SmartTurn、server_vad 或 endpointing 算法。
  • 不增加 SQLite、数据库、云同步、音频落盘或转写持久化。
  • 不实现 hard block、会话封禁、多 Owner、说话人分离、回放/克隆/活体检测。
  • 不改变日常聊天的声纹判定与上传行为。
  • 安全存储不可用时只允许本次内存使用,绝不明文降级。

验证结果

  • voice-identity 与 Independent ASR 定向回归:211 passed,1 skipped。
  • 本地持久化覆盖:重启恢复、密文篡改、错误密钥、安全存储不可用、原子替换、删除清理。
  • 录入覆盖:录入期间 Provider 收到零音频;自由测试未全部通过不替换现有 Profile;取消/异常释放临时状态。
  • Playwright 使用 fake microphone 验证 Web/Electron:完整录入、取消、删除、关闭钩子、过滤开关及窗口控制。
  • 8 个 locale key 对齐;新增 UI 无缺失翻译。
  • JavaScript 语法检查与 git diff --check 通过。

Regression Report

Scope

变更只触及本地 Owner Profile、录入控制面和设置 UI。运行时新增的音频 gate 仅在用户明确进入录入 session 且持有 lease 时生效,结束后恢复原路径。

Risk

  • 启动恢复: Profile 加载失败、密钥错误、文件损坏和模型不兼容均 fail-safe 为“无可用 Profile”,不阻断应用启动。
  • 音频隔离: 录入 lease 在 Provider 分流之前暂停普通 ASR/Omni 上传并清理遗留候选;录入 PCM 不进入 Provider。
  • 异步主循环: 文件、密钥和 CAM++ 计算均卸载到异步主循环之外。
  • 本地 API: 录入端点保持 local-only 与 CSRF 约束,不接受客户端文件路径。

Provider Matrix

  • OpenAI / Qwen / Soniox / Grok / Step:provider-native 路径未修改。
  • GLM / Gemini:SmartTurn 路径未修改。
  • 录入 lease 位于公共输入控制层,不进入任何 Provider worker。

Web / Electron

同一路由和向导状态机覆盖 Web 与 Electron;Electron 单窗口最小化、最大化和关闭行为已验证。页面关闭会取消未完成录入并释放临时 embedding。

Persistence / Privacy

仅保存 normalized embedding、模型标识、维度、profile revision、格式版本和必要时间;不保存 PCM、录音文件、转写文本、固定文案或相似度历史。密钥不与密文 Profile 以明文相邻存放。

@MomiJiSan
MomiJiSan requested a review from wehos as a code owner July 24, 2026 03:44
@coderabbitai

coderabbitai Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)
  • steam
  • react-rewrite
  • plugin

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 56c6ea24-7e34-4141-95da-915e77a4204e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

此 PR 增加 Owner 声纹录入与本地 Profile 管理喵。

  • 新增 AES-GCM 加密 ProfileStore、操作系统安全存储密钥及启动恢复流程喵。
  • 新增录入状态机、独占音频 lease、运行时 Profile 激活与本地 API 喵。
  • 新增 Web/Electron 设置页面、录音向导、过滤开关及 8 个 locale 的翻译喵。

Confidence Score: 5/5

在本次跟进审查允许的既有线程范围内,没有剩余阻塞问题,PR 看起来可以安全合并喵。

没有符合跟进审查资格且需要阻止合并的失败喵。

Important Files Changed

Filename Overview
main_logic/voice_identity/enrollment_service.py 新增 Owner 录入状态机、持久化提交、运行时激活、回滚、删除及资源清理流程喵。
main_logic/voice_identity/profile_store.py 新增 AES-GCM Profile 持久化、OS keyring 密钥管理、兼容性校验及原子文件替换喵。
main_logic/core/asr_runtime.py 新增录入期间的 PCM gate、ASR 中止以及已注册 Profile 的运行时同步喵。
main_routers/voice_identity_router.py 新增受本地访问与 mutation 校验保护的声纹录入、删除和过滤设置 API 喵。
static/js/voice_identity.js 新增浏览器与 Electron 共用的录音、录入向导和 Profile 管理交互喵。
templates/voice_identity.html 新增 Owner 声纹设置页面结构及录入控制界面喵。

Sequence Diagram

sequenceDiagram
  participant UI as Voice Identity UI
  participant API as Voice Identity Router
  participant ES as Enrollment Service
  participant PS as Encrypted Profile Store
  participant RT as ASR Session Managers
  UI->>API: Start enrollment
  API->>ES: Acquire enrollment lease
  ES->>RT: Fence ordinary microphone input
  UI->>API: Submit fixed/free PCM segments
  API->>ES: Build and verify embedding
  UI->>API: Commit
  ES->>PS: Encrypt and persist profile
  ES->>RT: Activate registered profile
  ES-->>API: Release lease and return status
  API-->>UI: Updated profile state
Loading

Reviews (3): Last reviewed commit: "feat(ui): 增加 Owner 声纹设置页面" | Re-trigger Greptile

Comment thread main_logic/voice_identity/enrollment_service.py
Comment thread main_logic/voice_identity/enrollment_service.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2316db40c1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread main_logic/voice_identity/registry.py Outdated
Comment on lines +44 to +47
def next_profile_revision(self) -> int:
with self._lock:
profile = self._profile
return 0 if profile is None else profile.profile_revision + 1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep profile revisions monotonic across deletion

After a profile has been activated, deleting it clears the registry but leaves each manager's VoiceIdentitySession alive with its previous _last_profile_revision. This method then restarts the next enrollment at revision 0; activating that replacement fails the session's profile_revision <= _last_profile_revision check, so the commit rolls back and returns voice_identity_unavailable. Thus, delete-then-re-enroll is unusable until the server restarts; retain a monotonic revision counter even when the current profile is cleared.

Useful? React with 👍 / 👎.

Comment on lines +398 to +402
async def set_filter_enabled(self, enabled: bool) -> EnrollmentStatus:
async with self._lock:
self._ensure_open()
self._registry.set_filter_enabled(enabled)
return self._status_locked()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Connect the filter toggle to ASR admission

Enabling this flag only changes registry/UI state. A repo-wide search shows filter_enabled is never read by the ASR or detector path, while the installed speaker runtime is explicitly observation-only, so every candidate is still forwarded to the provider exactly as when the toggle is off. Users enabling the advertised Voice filter therefore receive a successful response but no reduction in non-Owner ASR usage; wire the result into the admission decision (with the documented fail-open cases) or do not expose the toggle.

Useful? React with 👍 / 👎.

@MomiJiSan
MomiJiSan marked this pull request as draft July 24, 2026 04:18
@MomiJiSan
MomiJiSan force-pushed the codex/voice-owner-enrollment branch from 2316db4 to 65c5705 Compare July 24, 2026 04:22
@MomiJiSan
MomiJiSan marked this pull request as ready for review July 24, 2026 04:23
Comment thread main_logic/voice_identity/enrollment_service.py
Comment thread main_logic/voice_identity/enrollment_service.py

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 65c5705404

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +333 to +337
self._registry.install_profile(profile)
await self._activate_profile(profile)
except Exception as exc:
if persisted:
await self._rollback_activation(session, profile)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid retaining a stale profile revision after rollback

When activation fails after a successful save, _rollback_activation activates the previous profile at failed_profile.profile_revision + 1, but the still-active enrollment session retains profile at the older revision. A retry of /enrollment/commit therefore reinstalls and activates that stale profile; every already-updated VoiceIdentitySession rejects it because revisions must strictly advance, so a transient activation failure makes the session uncommittable until the user cancels and re-enrolls. Update or discard the temporary profile/session after rollback so a retry cannot reuse the obsolete revision.

Useful? React with 👍 / 👎.

Comment on lines +389 to +390
self._registry.clear_profile()
await self._activate_profile(None)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not clear the registry before deactivation succeeds

If deactivation fails in any character manager (for example, its verifier runtime cannot close), this clears the registry and reports the profile as deleted before _activate_profile(None) raises. The failing manager can retain its already-created verifier in the live ASR runtime, while subsequent status calls say no profile exists and the persisted recovery copy/key has already been removed. Deactivate all managers successfully before clearing the registry, or force-close/otherwise reconcile failed managers before returning deletion success/state.

Useful? React with 👍 / 👎.

@MomiJiSan
MomiJiSan force-pushed the codex/voice-owner-enrollment branch from 65c5705 to 479b6e6 Compare July 24, 2026 04:32
@MomiJiSan
MomiJiSan marked this pull request as draft July 24, 2026 04:40
@MingTianSang

Copy link
Copy Markdown
Contributor

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 479b6e6e4c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +194 to +197
account = hashlib.sha256(
str(self._path.parent.resolve()).encode("utf-8")
).hexdigest()
self._protector = protector or KeyringSecretProtector(account=account)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep the profile key account stable across storage moves

When a user moves the storage root, the existing migration code copies the whole config directory (utils/cloudsave_runtime/legacy_migration.py:139-145), so voice_identity.profile moves to a new parent path. Because the keyring account is derived from self._path.parent.resolve() here, the restored store looks under a different keyring account than the one used to encrypt the copied profile, creates/uses the wrong key, and loads the profile as invalid; users lose their enrolled Owner profile after a supported storage relocation. Use a stable per-installation account or explicitly migrate/delete the key with the profile.

Useful? React with 👍 / 👎.

elements.reenroll.disabled = state.busy || state.recording;
elements.delete.disabled = !state.profileAvailable || state.busy || state.recording;
elements.filter.checked = state.filterEnabled;
elements.filter.disabled = !state.profileAvailable || state.busy || state.recording;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Hide the filter switch until it affects ASR routing

When a profile exists, this line enables the filter switch and /api/voice-identity/filter accepts enabled: true, but a repo-wide search for filter_enabled/filterEnabled shows no ASR routing or verifier path reads that flag—only status/router/tests do, while the speaker shadow is created whenever a profile exists. Users can therefore turn on the advertised ASR-saving filter and still have all audio routed exactly as before; leave the control disabled/hidden until the soft gate is wired.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants