Skip to content

Commit 518a67c

Browse files
ghirparaclaude
andcommitted
Add webcam_expression_mirror.py — live FACS expression mirroring
Streams webcam face landmarks via MediaPipe to a Genesis 9 figure's FACS morph controls at up to 10 fps. Includes EMA smoothing, a live AU bar chart overlay, headless mode, and automatic morph reset on exit. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 1b137fd commit 518a67c

2 files changed

Lines changed: 487 additions & 0 deletions

File tree

docs/examples/README.md

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ The examples are roughly ordered from simple to complex. Start with
3030
| [dataset_generator.py](#dataset_generatorpy) | ML / Data | Generates a randomised render dataset with JSON sidecar for LoRA training | Yes |
3131
| [pose_interpolation.py](#pose_interpolationpy) | Animation | Interpolates between two saved states with easing curves and renders each step | Yes |
3232
| [expression_transfer.py](#expression_transferpy) | AI / Vision | Extracts a facial expression from a photo using MediaPipe and applies it to a Genesis 9 figure | No |
33+
| [webcam_expression_mirror.py](#webcam_expression_mirrorpy) | AI / Vision | Mirrors your live webcam expression onto a Genesis 9 figure in real time | No |
3334

3435
BVH / motion-capture examples (`bvh_import.py`, `bvh_discover.py`,
3536
`bvh_bone_maps.py`) are under active development and not yet stable.
@@ -345,3 +346,47 @@ python expression_transfer.py photo.jpg --debug
345346
| `--list-properties` | off | List all numeric properties on the figure and exit |
346347
| `--search TERM` || Filter `--list-properties` output by case-insensitive substring |
347348
| `--debug` | off | Print which FACS labels matched/missed and suggest candidates |
349+
350+
---
351+
352+
### webcam_expression_mirror.py
353+
354+
Captures frames from your webcam, runs MediaPipe FaceLandmarker on each frame,
355+
and streams the resulting FACS morph values to a Genesis 9 figure at up to
356+
`--fps` updates per second. The figure's expression updates live as your face
357+
moves.
358+
359+
Pairs naturally with DAZ's **Face Transfer 2**: use that tool to build a 3D
360+
version of yourself, then use this script to drive its expressions in real time.
361+
362+
Press **Q** in the preview window, or **Ctrl+C**, to stop. FACS morphs are
363+
zeroed on exit so the figure returns to a neutral expression.
364+
365+
**Dependencies** (in addition to `dazpy`):
366+
```bash
367+
pip install mediapipe opencv-python numpy
368+
```
369+
370+
The MediaPipe face landmarker model (`face_landmarker.task`, ~1 MB) is
371+
downloaded automatically to the `docs/examples/` directory on first run.
372+
373+
**FACS label calibration:** Uses the same `FACS_MAP` as `expression_transfer.py`.
374+
Run `expression_transfer.py --list-properties` to discover labels for your
375+
installed FACS product if morphs don't apply.
376+
377+
```bash
378+
python webcam_expression_mirror.py
379+
python webcam_expression_mirror.py --figure "Genesis 9" --scale 0.8
380+
python webcam_expression_mirror.py --camera 1 --fps 15
381+
python webcam_expression_mirror.py --smooth 0.7
382+
python webcam_expression_mirror.py --no-preview
383+
```
384+
385+
| Argument | Default | Description |
386+
|---|---|---|
387+
| `--figure LABEL` | `Genesis 9` | Target figure label |
388+
| `--scale FLOAT` | `1.0` | Global expression scale factor |
389+
| `--camera N` | `0` | OpenCV camera index (try `1`, `2`, … for external webcams) |
390+
| `--fps N` | `10` | Max DAZ Studio updates per second |
391+
| `--smooth FLOAT` | `0.5` | EMA smoothing: `0` = raw/responsive, `0.9` = very smooth |
392+
| `--no-preview` | off | Run headless with no OpenCV window (Ctrl+C to stop) |

0 commit comments

Comments
 (0)