Lighting problems both backends

[cpinter]

Details here:
https://projectweek.na-mic.org/PW42_2025_GranCanaria/Projects/LightingProblemsWithLatestSlicervr/

[jcfr]

From @sankhesh

Regarding washed out lighting, that is most likely related to differences in color scaling.
It seems some people have reported issues with OpenXR creating an sRGB framebuffer. Could you please add this->GetState()->vtkglDisable(GL_FRAMEBUFFER_SRGB) in the bind framebuffer function of openxr render window?

Regarding VR view, have you tested to ensure that the appropriate changes are propagated to the vtk render window and mappers?

[jcfr]

From @LucasGandel

The issue could be related to sRGB vs RGB color space support.

Some mentions of similar issues:

• https://www.youtube.com/watch?v=oxTA_SCHC4U
• OpenXR/OpenGL(ES) rendering has washed out colors on some android devices godotengine/godot#74857
• https://communityforums.atmeta.com/t5/OpenXR-Development/sRGB-RGB-giving-washed-out-bright-image/td-p/957475

I don't think we handle sRGB support in the current VTK OpenXR implementation.
It would be great to try a few things:

• Enable UseSRGBColorSpaceOn on vtkOpenXRRenderWindow and its the internal HelperWindow.
• Check the values returned by vtkOpenXRManagerOpenGLGraphics::GetSupportedColorFormats() and maybe remove everything but GL_SRGB8_ALPHA8_EXT
• Enable UseSRGBColorSpace on the internal textures of the render window
• Perform gamma correction manually in the shader code

This is just a guess, and I'd love to hear what @sankhesh thinks.
If we confirm this is related to sRGB support we can then focus on fixing it properly.

[LucasGandel]

I gave it a quick try and the only solution I found to fix the colors is to remove all the requested formats but GL_SRGB8_ALPHA8_EXT in vtkOpenXRManagerOpenGLGraphics::GetSupportedColorFormats().
OpenXR seems to perform an automatic sRGB-to-linear conversion in this case according to: https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#rendering-swapchain-image-management

The right fix is probably to put this value first in the list and revert the logic of the selectPixelFormat lambda function in vtkOpenXRManager.cxx

[cpinter]

Thanks a lot @LucasGandel for investigating! Sounds great!

[cpinter]

I tried the OpenXR backend with the changes suggested by @LucasGandel and unfortunarely I don't see any change in the colors after removing the other formats and just leaving GL_SRGB8_ALPHA8_EXT.

Lucas, can you confirm that it in fact worked for you? I double checked that the application uses my changes but the rendering is the same. If it worked for you, then any idea what might have changed and why it doesn't make a difference now?

Until I get a response I'll keep investigating. Thanks a lot!

[cpinter]

Just recording here what I have found:

This is what the image should look like (shadows seem to be always off in the VR view and cannot be turned on):

This is what it looks like, with the original list of color formats, or if I only leave GL_SRGB8_ALPHA8_EXT in the list (I tried to use other SRGB formats like GL_SRGB8_ALPHA8 or GL_SRGB, it is the same):

If I do this in the code:

Enable UseSRGBColorSpaceOn on vtkOpenXRRenderWindow and its the internal HelperWindow
then I get this:

(this last view is the same if I only have SRBG color formats in the list or not)

[cpinter]

At the Slicer project week we did some testing with @Sunderlandkyl and @lassoan and it seems that we don't have this specific problem anymore. Any remaining issue might be due to lighting settings. I'll ask the SlicerHeart group to do some testing and we can close this once they confirm.