Skip to content

Automatically use zip storage if the .ozx output file extension is used#322

Draft
melissalinkert wants to merge 5 commits into
glencoesoftware:masterfrom
melissalinkert:zipstore
Draft

Automatically use zip storage if the .ozx output file extension is used#322
melissalinkert wants to merge 5 commits into
glencoesoftware:masterfrom
melissalinkert:zipstore

Conversation

@melissalinkert

Copy link
Copy Markdown
Member

Fixes #164, see https://ngff.openmicroscopy.org/rfc/9/index.html. Not super high priority to review, just had a branch in progress that I didn't want to forget about.

This required updating to zarr-java 0.1.3 due to zarr-developers/zarr-java#73. A side effect of that was needing to replace all usage of byteshuffle with shuffle, but that seems to be expected based on https://github.com/scalableminds/blosc-java/compare/ca183797fcb7df2ae61323141108235a41f62076..ubuntu-arm64#diff-7b6ef77c00daf6ad090dbc1d01878b090b49803117075a1b0b24f1dbde000bbeR259 (cc @normanrz).

Conversion behavior should be unchanged with default NGFF version (0.4, i.e. Zarr v2). If --ngff-version 0.5 is used and the output file name ends with .ozx, then the resulting output should be a single file (not a directory) that contains the whole dataset. raw2ometiff does not yet support the .ozx file as input, but unzipping the .ozx file to its own .zarr directory and running raw2ometiff on that directory should work as normal. Using --ngff-version 0.5 without the .ozx extension should also continue to work as normal.

I did need to change a bit how the OME/METADATA.ome.xml file is written, since everything now needs to be written through the Store instance or it won't end up in the .ozx file. When trying the non-zip output, it would be good to double-check that the OME/METADATA.ome.xml file still looks fine.

I think we could technically use this feature with NGFF 0.4/Zarr v2, but RFC-9 doesn't apply to that version of the spec, so maybe that's making things too complicated.

I did not extensively test this with larger datasets, so would be good to include some larger slides and plates when doing functional testing, in particular things that might cause the output file to exceed 4GB.

@melissalinkert melissalinkert added this to the 0.13.0 milestone May 5, 2026
@melissalinkert melissalinkert requested review from erindiel and sbesson May 5, 2026 01:53
@melissalinkert melissalinkert marked this pull request as draft May 13, 2026 15:07
@melissalinkert

Copy link
Copy Markdown
Member Author

b7a789d is split into #327; I have not removed the commit here since I'd expect the test to fail without it. I can rebase this branch once #327 is merged.

@melissalinkert

Copy link
Copy Markdown
Member Author

Updated since #327 is merged. As with #330, this should likely be using --ngff-version 1.0-DEV, so can update this once #330 is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ZipStore as an output option

1 participant