Skip to content

[Bug]: Fullscreen Proton/Wine games always render to the physical monitor, never to the Polaris capture target (headless or virtual) #234

Description

@mrba11s

Describe the bug

No matter what capture mode I use, any fullscreen 3D Windows game launched through Proton/Wine renders to my real physical monitor instead of whatever Polaris is actually capturing (headless labwc session, or a Host Virtual Display). Audio, input, framerate limits, and per-app env vars all correctly reach the isolated session — it's specifically the video output that escapes. I've tested this across two completely different games and two completely different Proton/launcher stacks, so I don't think it's a launcher quirk.

Steps to reproduce

Set up an app entry in Polaris pointing at a fullscreen-capable Windows game via Proton (tested with both Faugus and Lutris as launchers — see below).
Launch the app from a Moonlight-compatible client (Nova or Moonlight).
Observe: the game opens and renders on the host's real physical monitor. The client stream shows a black screen (or an empty desktop, depending on capture mode) with no game visible. Audio, however, streams correctly to the client throughout.

This happens consistently regardless of capture mode:

Private Stream (Recommended) — headless labwc, linux_use_cage_compositor = enabled, linux_prefer_gpu_native_capture = disabled (the config recommended in troubleshooting.md under "Headless session does not start cleanly")
Private Stream (GPU Native)
Host Virtual Display (real second xrandr output, e.g. Virtual-virtual-xdp-kde- at 1920x1080+2560+0)
Mirror Desktop

Games/launchers tested (same result both times):

The First Berserker: Khazan, launched via Faugus (Flatpak) → umu-run → Proton (cachyos_11.0_20260702-LinUwUx)
A separate game launched via Lutris (Flatpak, net.lutris.Lutris lutris:rungameid/1), different Proton build entirely

Install method

Arch package

Host OS

CachyOS, KDE Plasma, Wayland

GPU and driver

CPU: Ryzen 9 9800X3D GPU: AMD Radeon RX 9070 XT (RDNA4, gfx1201) Mesa 26.1.5-arch3.1, driver=amdgpu, radv

Client and device

Nova on a Lenovo Legion Y700 Gen 3 (Snapdragon 8 Gen 3) Same result reproduced on a second client (HP ProDesk running Moonlight)

Runtime path

None

Expected behavior

The fullscreen game should render inside whatever session Polaris is capturing (the headless labwc session, or the Host Virtual Display), so it streams correctly to the client — the same way the desktop and windowed-mode apps already do in my testing.

Relevant log output

WAYLAND_DISPLAY and DISPLAY are confirmed correct on the actual game process itself via /proc/<pid>/environ (WAYLAND_DISPLAY=wayland-1, DISPLAY=:2), not just on the launcher wrapper, so the environment is reaching the process correctly:

WAYLAND_DISPLAY=wayland-1
DISPLAY=:2

Polaris log snippet from a headless launch showing correct session setup, correct env injection, and correct capture backend selection, right up until the point the video should appear on the client:

Info: cage: Set WAYLAND_DISPLAY=wayland-1 DISPLAY=:2 for app command
Info: Executing: [/home/andy/khazan_launch.sh] in ["/home/andy"]
...
Info: Using SHM screencopy (preferred for this runtime)
Info: SHM screencopy buffer allocated: 2560x1440 stride=10240 format=875709016
Info: Wayland virtual input: routing supported devices to labwc socket [wayland-1]
Warning: wlr: capture_transport=shm frame_residency=cpu frame_format=bgra8; capture will incur an extra CPU-side copy/conversion path
Info: Client requested cursor visibility change: visible
...
Info: PipeWire stream is now actively streaming audio

Happy to attach a full support bundle if useful.

Screenshots or recordings

No response

Additional context

Wrapped the launch in gamescope --backend wayland to force nesting inside the labwc session. Confirmed via gamescope's own log that it correctly initialized a nested Wayland backend on the intended socket — game still escaped to the physical monitor.
Tested in true headless mode (single output, HEADLESS-1, no second display in the session at all) to rule out any "wrong primary monitor" selection — same result. This rules out the known Proton/Wine bug where fullscreen games attach to whichever display is marked primary (ValveSoftware/Proton#6938), since there's nothing else for it to attach to in a single-output headless session.
Manually forced the game into windowed mode via its own settings, at which point the window became genuinely draggable with no title bar (Alt+drag worked) — I was able to physically drag it onto the Host Virtual Display and switch it to borderless windowed there, and it streamed to the client correctly. This confirms the compositor/capture pipeline itself works fine; it's specifically fullscreen mode that escapes.
Edited the game's GameUserSettings.ini directly (WindowPosX/Y, DefaultMonitorIndex, FullscreenMode=1 for borderless) to try to force it onto the virtual display from launch — had no effect, the game still opened on the physical monitor regardless of these values.
Checked DXVK's documented behavior — by default DXVK disables Vulkan exclusive fullscreen entirely on Linux, so if this is a DX11 title it shouldn't be able to bypass the compositor this way at all, which makes me suspect this may be VKD3D-Proton (DX12) specific behavior instead, though I haven't been able to confirm that further.
Would appreciate knowing if this has been reported with other fullscreen titles, particularly on RDNA4/gfx1201 hardware, and whether there's a known-good reference title/Proton build I could test against to isolate hardware/driver vs. Polaris capture logic vs. Proton itself.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingquestionFurther information is requested

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions