Uniform buffers for shader props, allow variable number of channels, update deck.gl to 9.2.9 by xinaesthete · Pull Request #924 · hms-dbmi/viv

xinaesthete · 2025-11-08T18:22:09Z

Fixes #687

Background

#687 as well as elsewhere for discussion of UBO etc.

This is work-in-progress... started out just trying to update to latest luma.gl/deck.gl to deal with some peer-dependency issues downstream. Latest versions remove the deprecated setUniforms etc, so this is becoming more pressing.

This is still WIP as of this writing, but I think starting to take shape for a way of supporting N-channels without too much pain for extension authors.

Change List

declaring shader uniforms as UBO
setting props with model.shaderInputs rather than setUniforms
tending to prefer implementing updateState() rather than draw(), I think this is more in line with how things are supposed to be
trying to make sure any assumptions about MAX_CHANNELS = 6 is findable in the code etc...
shaders have a define for NUM_CHANNELS which should be used consistently where relevant
initial version of a shader code preprocessor wrapped in a new VivShaderAssembler class for expanding lines with <VIV_CHANNEL_INDEX>...

Checklist

Update JSdoc types if there is any API change.
Make sure Avivator works as expected with your change.
Update documentation for migrating shader extensions to the new form once the design is settled.

…ean. some tests are failing with this revision, runtime not tried.

… i is clean." This reverts commit c51bffe.

…els allowed hms-dbmi#687

dbmi-svc-checkmarx · 2025-11-08T18:24:25Z

Checkmarx One – Scan Summary & Details – 530f1bc2-c855-4937-8261-34d77ee495ea

New Issues (3)

Checkmarx found the following issues in this Pull Request

#	Issue	Source File / Package	Checkmarx Insight
1	Unpinned Actions Full Length Commit SHA	/ci.yml: 39	details Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA help...
2	Unpinned Actions Full Length Commit SHA	/ci.yml: 32	details Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA help...
3	Unpinned Actions Full Length Commit SHA	/release.yml: 34	details Pinning an action to a full length commit SHA is currently the only way to use an action as an immutable release. Pinning to a particular SHA help...

Fixed Issues (3)

Great job! The following issues were fixed in this Pull Request

Severity	Issue	Source File / Package
	~~Unpinned Actions Full Length Commit SHA~~	/release.yml: 33
	~~Unpinned Actions Full Length Commit SHA~~	/ci.yml: 37
	~~Unpinned Actions Full Length Commit SHA~~	/ci.yml: 30

Use @Checkmarx to interact with Checkmarx PR Assistant.
Examples:
@Checkmarx how are you able to help me?
@Checkmarx rescan this PR

ilan-gold

I think it's reasonable to ask in deck.gl slack or on the github what the implications of a custom ShaderAssembler would be. I really couldn't know off hand. As you can probably tell, I did a lot of this by just looking at the underlying code and ballparking what would be needed. The upside is it works. The downside is that it has a bunch of private accesses. At the time, I was under paper pressure :/

I don't feel too strongly about how off-the-beaten-path this goes as long as it works. We advertise breaking changes on the minor version so I'm also not worried about any breaking changes. I'll try to download the deck.gl slack on Monday again.

ilan-gold · 2025-11-08T18:53:34Z

packages/extensions/src/additive-colormap-3d-extensions/additive-blend-extension.js


 const _RENDER = `\
-  float intensityArray[6] = float[6](intensityValue0, intensityValue1, intensityValue2, intensityValue3, intensityValue4, intensityValue5);
+  float intensityArray[NUM_CHANNELS] = float[NUM_CHANNELS](intensityValue0, intensityValue1, intensityValue2, intensityValue3, intensityValue4, intensityValue5);


Yes, this is great!

Note that there is one other place where I have something more like

float array[NUM_CHANNELS] = float[NUM_CHANNELS]( value<VIV_CHANNEL_INDEX>, );

and that also works...

Now I'm trying to get that to also work for uniform vec2 constastLimits<VIV_CHANNEL_INDEX>; with the new shaderInputs UBO binding and finding it tricky.

Or rather... the syntax I mentioned works ok, but the new

uniform xrLayerUniforms { vec2 contrastLimits<VIV_CHANNEL_INDEX>; } xrLayer;

I'm struggling with.

I think the issue may be that I was trying to implement setting shaderInputs in the draw() method and moving this to updateState() makes a significant difference (just about to try this). edit: also, working in js rather than ts and having typos.

ilan-gold · 2025-11-08T18:57:19Z

packages/extensions/src/additive-colormap-extension/additive-colormap-extension.js

+    // does any of this existing logic need to change?
+    // (like, is there ever more than one model?)


I think the way our extensions are structured, no, because extension is attached to XRLayer which does its own rendering?

Yes I don't anticipate a problem there... In some places I'm looping over this.getModels() and it may be that it would be somewhat safer to do that a bit more consistently, but I'm pretty sure it isn't likely to matter in foreseeable future. Slight balance between consistency and avoiding changes where there isn't a solid reason for it.

xinaesthete · 2025-11-08T19:19:58Z

@ilan-gold Ib already expressed some enthusiasm about viv updating deck versions and I'm sure he'll be happy to give some more feedback at some point. I'll add a pointer to this PR over there now.

xinaesthete · 2025-11-10T08:10:14Z

n.b. I've made all the CI pass and will try to keep commits clean WRT that, but bear in mind that I don't think the configuration is properly working for '@deck.gl/test-utils' as noted #921 so we need to be extra careful to manually verify that layers/extensions etc do actually work correctly.

xinaesthete · 2025-11-10T08:48:06Z

Also I think that volume minimum intensity projection is already buggy - it looks similar in this branch to production avivator build, which is to say, not good.

…e to 10 for testing.

ilan-gold · 2025-11-10T16:00:43Z

Also I think that volume minimum intensity projection is already buggy - it looks similar in this branch to production avivator build, which is to say, not good.

Yes it is, should eventually fix that...

keller-mark · 2025-11-10T16:14:29Z

I am not sure that minimum intensity projection has bugs, it is just that it is not really useful, conceptually, for most images. I think it was implemented because it is very easy once you implement the max-intensity projection. But why would I want to view the "minimum-intensity" pixels, when they are just going to correspond the black space that I am not typically interested in?

I think the only time it would really make sense is if the image colors have been inverted for some reason.

(Maybe I am wrong, but before we assume it has bugs we should verify with another viewer that supports MinIP)

ilan-gold · 2025-11-10T16:17:29Z

You're right @keller-mark I was mostly speaking from an experience where I wanted the inversion actually and remember it not working properly. But I don't know for sure.

xinaesthete · 2026-03-05T15:01:15Z

Documentation probably still needs some work.

add helper function for better type in test

…er exist

…te in bitmap-layer

xinaesthete · 2026-03-09T17:32:38Z

There seems to be a fairly major performance degradation with volume rendering currently – I would expect it to be faster than before as long as it's rendering a lower number of channels, and I believe this was the case previously, so I'll try to figure out what's going on there.

xinaesthete · 2026-03-10T10:14:30Z

There seems to be a fairly major performance degradation with volume rendering currently – I would expect it to be faster than before as long as it's rendering a lower number of channels, and I believe this was the case previously, so I'll try to figure out what's going on there.

edit: I have noticed that it seems much worse for camera interactions than tweaking contrast-limits. At least with my local changes there are intermittent pauses when moving the camera, so this is something to investigate; seems like the shader details are not so relevant. I might post on the deck.gl Slack.

Also, I think I've been seeing some stutters in 2D as well, but not on the same level.

older comments

I've realised I've got channel-expansion to arrays inside the hot-path of the raymarch loop in the volume shader... whoops... ~~that'd likely explain some performance degradation~~. I think this is actually all pretty trivial these days and I'm not seeing much change from hoisting things out of loops etc; I expect somewhere up the GLSL compiler stack it's able to do that automatically.

~~It seems that there are calls being made to bufferSubData every frame which surely shouldn't be necessary~~. Wasn't thinking straight there – this is the normal expected UBO updating, I don't think I've seen texture data being thrashed.

Hopefully it's just a case of tweaking updateState / draw etc.

…sible, fixes bug with always selecting default colormap

ilan-gold · 2026-03-19T16:00:08Z

@xinaesthete Apologies, was probably https://github.com/xinaesthete/viv/blob/deck_9.2/packages/viewers/src/VivViewer.jsx#L73-L75 which doesn't actually seem to work on everyone's machines/site builds.

I reverted it in #948 but am waiting on @keller-mark there. Maybe I should just revert globally. Sorry for the trouble!

sites/avivator/src/index.css

commit 91560ab Author: Ilan Gold <ilanbassgold@gmail.com> Date: Thu Mar 19 17:22:12 2026 +0100 fix: remove debounce (hms-dbmi#952)

xinaesthete · 2026-03-31T06:15:35Z

.github/workflows/ci.yml

      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
+        with:
+          version: 9.5.0


I changed these because at one point I didn't realise that CI on the branch was running on an automatic merge with main and I was trying to figure out why CI was failing when everything worked locally. package.json has "packageManager": "pnpm@9.5.0", there may be something to be said for making sure that these are all tied to the same version, but the change isn't necessary to the PR (CI was breaking because of silently auto-merged pnpm-lock.yml I think), and if we make a separate change it'd probably also update to a newer version.

xinaesthete added 9 commits November 8, 2025 12:09

update deck and luma versions, with changes to make sure pnpm i is cl…

c51bffe

…ean. some tests are failing with this revision, runtime not tried.

Revert "update deck and luma versions, with changes to make sure pnpm…

fa0df0e

… i is clean." This reverts commit c51bffe.

refactor additive-colormap-extension to UBO

26e6570

change BaseExtension draw to updateState

b702528

rewrite shaders to use NUM_CHANNELS (still very much fixed) Max chann…

671332f

…els allowed hms-dbmi#687

move NUM_CHANNELS define to default module

9d64352

more consistent use of NUM_CHANNELS, some comments

f445989

add VivShaderAssembler, fix volume renderer missing MAX_CHANNELS

8eb8c6e

first pass at shader-preprocess

756d33a

xinaesthete changed the title ~~Deck 9.2~~ Uniform buffers for shader props, allow variable number of channels Nov 8, 2025

ilan-gold reviewed Nov 8, 2025

View reviewed changes

xinaesthete added 7 commits November 9, 2025 07:46

mostly comments and inactive code for attempting contrastLimits UBO

eaa1ea5

use channel index template for volume shader

e1d1916

use imported template string in xr-fragment

973223a

start trying to refactor towards tagged-template for shaders

88c8719

add missing MAX_CHANNELS import in bitmap-layer.js

5fe5de4

biome fix, some comments

e4d44d3

biome fix

64b1da7

xinaesthete added 5 commits November 10, 2025 09:01

move VIV_CHANNEL_INDEX_PLACEHOLDER to constants

27486ec

more consistent application of VIV_CHANNEL_INDEX_PLACEHOLDER

27935fb

Make avivator use value of MAX_CHANNELS from constants package. Chang…

2807bfd

…e to 10 for testing.

biome fix

2d6c43d

make tests pass with 10 channels

38dc474

xinaesthete added 3 commits March 5, 2026 10:54

update pnpm-lock

373cfa6

tests for extensions

8fb5cfd

lint

d1f3846

xinaesthete marked this pull request as ready for review March 5, 2026 15:00

xinaesthete added 4 commits March 6, 2026 11:33

remove redundant uniformTypes in getRenderingAttrs

36fad0a

add helper function for better type in test

extension tests use getShaders rather than expandShaderModule

f68540c

lint

f5c1711

update documentation

04282b0

xinaesthete marked this pull request as draft March 6, 2026 17:05

remove redundant test of getRenderingAttrs uniformTypes which no long…

ba8482d

…er exist

xinaesthete mentioned this pull request Mar 9, 2026

Out of memory errors #949

Closed

xinaesthete added 2 commits March 9, 2026 10:54

fix memory errors hms-dbmi#949 by disabling pool again

489bc72

avoid mutating props, make sure to call super.updateState/finalizeSta…

22b46bc

…te in bitmap-layer

xinaesthete marked this pull request as ready for review March 9, 2026 13:46

xinaesthete added 3 commits March 9, 2026 13:51

remove numTextures which was always equal to numChannels

a0ae21f

fix example VivLayerExtension in doc

0c0fc77

change extension example to spread rest props in updateState

b4e518a

xinaesthete added 3 commits March 10, 2026 13:48

call to getVivShaderTemplates binds this to make sure props are acces…

bb82638

…sible, fixes bug with always selecting default colormap

using line-expansion rather than arrays and loops for 3d shaders

9c1ab30

refactor xr-3d-layer particularly to avoid GC

8012713

Merge branch 'main' into deck_9.2

5b65db7

xinaesthete commented Mar 19, 2026

View reviewed changes

sites/avivator/src/index.css Show resolved Hide resolved

xinaesthete added 2 commits March 19, 2026 16:32

lint & typo fix

d063d93

Squashed commit of the following:

dce96f4

commit 91560ab Author: Ilan Gold <ilanbassgold@gmail.com> Date: Thu Mar 19 17:22:12 2026 +0100 fix: remove debounce (hms-dbmi#952)

xinaesthete commented Mar 31, 2026

View reviewed changes

add changeset

92ea594

		// does any of this existing logic need to change?
		// (like, is there ever more than one model?)

Conversation

xinaesthete commented Nov 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Background

Change List

Checklist

Uh oh!

dbmi-svc-checkmarx commented Nov 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ilan-gold left a comment

Choose a reason for hiding this comment

Uh oh!

ilan-gold Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

xinaesthete Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

xinaesthete Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

xinaesthete Nov 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ilan-gold Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

xinaesthete Nov 8, 2025

Choose a reason for hiding this comment

Uh oh!

xinaesthete commented Nov 8, 2025

Uh oh!

xinaesthete commented Nov 10, 2025

Uh oh!

xinaesthete commented Nov 10, 2025

Uh oh!

ilan-gold commented Nov 10, 2025

Uh oh!

keller-mark commented Nov 10, 2025

Uh oh!

ilan-gold commented Nov 10, 2025

Uh oh!

xinaesthete commented Mar 5, 2026

Uh oh!

xinaesthete commented Mar 9, 2026

Uh oh!

xinaesthete commented Mar 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

older comments

Uh oh!

ilan-gold commented Mar 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

xinaesthete Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xinaesthete commented Nov 8, 2025 •

edited

Loading

dbmi-svc-checkmarx commented Nov 8, 2025 •

edited

Loading

xinaesthete Nov 8, 2025 •

edited

Loading

xinaesthete commented Mar 10, 2026 •

edited

Loading

ilan-gold commented Mar 19, 2026 •

edited

Loading