Skip to content

[VR] Quest 3 Stereo Rendering — Per-Eye Covariance, Shared Compute, Performance Fixes#225

Open
arghyasur1991 wants to merge 2 commits intoaras-p:mainfrom
arghyasur1991:feature/quest-stereo-rendering
Open

[VR] Quest 3 Stereo Rendering — Per-Eye Covariance, Shared Compute, Performance Fixes#225
arghyasur1991 wants to merge 2 commits intoaras-p:mainfrom
arghyasur1991:feature/quest-stereo-rendering

Conversation

@arghyasur1991
Copy link
Copy Markdown

@arghyasur1991 arghyasur1991 commented Mar 16, 2026

Supersedes #173 with a cleaner, more correct implementation. Includes all fixes from the discussion there.

Summary

Adds single-pass instanced and multiview stereo rendering support for Quest 3 (and other VR headsets using URP), building on the approach from #173 with key correctness and performance improvements:

  • Per-eye covariance projection: Passes explicit per-eye model-view and projection matrices to the compute shader via a StereoMatrices cbuffer. The original [VR] Single Pass Instanced/Multiview Support for Gaussian Splatting in URP #173 used UNITY_MATRIX_VP / _MatrixMV (center eye) for covariance math, causing visible artifacts ("thorny ferns") in VR. This fix computes correct 2D covariance per eye.
  • Shared covariance/SH between eyes: Computes full covariance, SH evaluation, and color once for the left eye, reuses for the right — only the clip position is recomputed. Saves ~40% of compute shader ALU.
  • Sort once, render twice: Sorting uses the center eye matrix (IPD offset is negligible for depth ordering), so per-eye sorting was removed — it produced identical results at 2x GPU cost.
  • clip() instead of discard in the fragment shader for better Adreno TBDR performance.
  • Dispatch fix: Uses m_SplatCount instead of m_GpuView.count for compute dispatch (view buffer is 2x for stereo).

Bug fixes (from #173 discussion)

  • Editor stereo guard: !Application.isEditor prevents false isStereo detection with MockHMD/OpenXR in editor play mode (fixes #173 comment — right eye rendering black)
  • Tex2DArray dimension check: Prevents stereo path for OVROverlayCanvas and other stereo-enabled but 2D-target cameras
  • RT dimension mismatch: Uses cameraTargetDescriptor instead of XRSettings.eyeTextureDesc for RT creation, fixing depth/color surface dimension errors on Quest
  • Sorter validity guard: m_Sorter.Valid check before dispatch for GPUs lacking wave intrinsics

Stereo rendering approach

The CSCalcViewData compute kernel now:

  1. Checks _IsStereo uniform
  2. If stereo: calls CalculateEyeViewData() with left-eye matrices (full covariance + SH), copies result for right eye with only the clip position recomputed from right-eye VP matrix
  3. Writes to _SplatViewData[idx * 2] (left) and _SplatViewData[idx * 2 + 1] (right)
  4. RenderGaussianSplats.shader indexes into view data via instID * 2 + eyeIndex

The URP feature detects stereo via XRSettings + Tex2DArray check, uses PrepareSplats() (sort + calc view data once) then RenderPreparedSplats() per eye, compositing through a stereo tex2darray composite shader.

What's NOT in this PR

  • HDRP stereo support
  • Debug point/box shader stereo variants
  • unity_StereoEyeIndex usage (doesn't work with DrawProcedural on Quest 3 — manual 2-draw-call workaround used instead, see TODO in code)

Tested on

  • Quest 3 (Multiview mode) — correct stereo rendering, stable frame rate
  • Desktop non-VR — no regressions, single-eye path unchanged

Single-pass instanced and multiview stereo rendering for Quest 3:
- Per-eye model-view and projection matrices in compute shader for
  correct covariance projection (fixes severe VR visual artifacts)
- Shared covariance/SH computation between eyes (compute once for
  left, reuse for right — only clip position differs)
- Sort once, render twice — sorting uses center eye matrix
- Stereo composite shader with tex2darray sampling per eye
- clip() instead of discard in fragment shader (Adreno TBDR perf)
- Dispatch based on splat count instead of view buffer count
- View data buffer doubled for stereo (left + right eye data)

Made-with: Cursor
- Skip stereo path in editor (!Application.isEditor) to prevent
  incorrect isStereo detection with MockHMD/OpenXR in play mode
- Check cameraTargetDescriptor.dimension == Tex2DArray to skip stereo
  for OVROverlayCanvas and other stereo-enabled-but-2D cameras
- Use cameraTargetDescriptor for RT creation instead of
  XRSettings.eyeTextureDesc (fixes depth/color dimension mismatch)
- Guard m_Sorter.Dispatch with m_Sorter.Valid for GPUs lacking wave
  intrinsics

Made-with: Cursor
@arghyasur1991
Copy link
Copy Markdown
Author

arghyasur1991 commented Mar 18, 2026

not raising PR yet, but there are more performance optimizations present in https://github.com/arghyasur1991/UnityGaussianSplatting/tree/feature/quest-stereo-perf and my fork's main branch which allows rendering gaussian splats count as high as 300k at 16-18FPS in quest 3 without any jittering/visible frame drops (even from closeups).
That branch includes high impact changes like -

  1. Early out culling based on opacity/scale and frustum - Medium impact without perceivable quality loss
  2. Reduced-resolution rendering with bilinear upscale - High impact without much perceivable quality loss
  3. Lower RT precision - High impact without perceivable quality loss

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant