v2.5.0

Brings region-writing support in .to_icechunk(), a ZarrParser with orders of magnitude better performance, more FAQ docs, and various bugfixes.

What's Changed

• v2.4.1 empty release notes by @TomNicholas in #857
• Fix fill_value=null zarr v2 to v3 conversion for string dtypes by @observingClouds in #845
• Use new obspec utils API by @TomNicholas in #863
• Skip empty chunks when parsing Kerchunk Parquet by @TomNicholas in #864
• Test that we forbid inlined refs when parsing kerchunk parquet by @TomNicholas in #865
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #866
• Fixes error when running with Zarr-Python 3.1.0 by @omshinde in #868
• [docs] remove extra comma in kerchunk example by @norlandrhagen in #871
• Correct Scaling docs. by @jbusecke in #890
• Update FAQ regarding virtualizing existing Zarr V2 data by @TomNicholas in #893
• Fix dmrpp error handling by @betolink in #880
• Allow running Zarr parser inside running event loop by @TomNicholas in #900
• Support functools.partial with lithops by @chuckwondo in #899
• Update kerchunk writer refs_to_dataframe to deal with ArrowStringArray from new ZarrParser deps. by @norlandrhagen in #907
• Skip path re-validation by @TomNicholas in #910
• Updates zarr-parser to use obstore list_async instead of concurrent_map by @norlandrhagen in #892
• Compile regular expressions by @chuckwondo in #909
• Fix docs in CEPH/OSN usage tab by @charles-turner-1 in #914
• Fix for nested store path in ZarrParser by @TomNicholas in #913
• fix: use public attribute in zarr parser by @maxrjones in #916
• Add faq answer comparing kerchunk format to icechunk format by @TomNicholas in #818
• feat: configurable chunk separator by @maxrjones in #917
• chore: caching and timeouts to protect against CI hangs by @maxrjones in #920
• chore: add support for Python 3.14 by @maxrjones in #831
• Implement to_icechunk(region=...) by @vladidobro in #873
• FAQ answer on "why still write native zarr?" by @TomNicholas in #918
• chore: add test for ZarrParser on bucket without list permissions by @maxrjones in #919
• Fix Coord Name Issue by @UserNobody14 in #924
• Remove pyarrow dependency from ZarrParser by @TomNicholas in #927
• Run minimum-versions CI on all PRs by @TomNicholas in #930
• Update min deps by @TomNicholas in #931
• Test python 3.12, 3.13 by @jbusecke in #932
• Mark lithops tests as flaky, run in separate CI job by @TomNicholas in #934
• Fix bug in v2 zarr parser which interprets empty dimension list as truthy by @jbusecke in #936
• R2 docs by @TomNicholas in #937
• Corrections to docs/releases.md by @TomNicholas in #940
• Add release notes for version 2.5.0 by @TomNicholas in #941

New Contributors

• @observingClouds made their first contribution in #845
• @omshinde made their first contribution in #868
• @betolink made their first contribution in #880
• @charles-turner-1 made their first contribution in #914
• @vladidobro made their first contribution in #873
• @UserNobody14 made their first contribution in #924

Full Changelog: v2.4.0...v2.5.0

v2.4.0

This release moves the ObjectStoreRegistry to a separate package obspec_utils, and provides a way to customize how files are read, which can easily allow open_virtual_dataset to run over ~5x faster.

What's Changed

• Move ObjectStoreRegistry and Reader functionality to obspec_utils by @maxrjones in #844
• Add example of virtualizing GOES using caching and request splitting by @maxrjones in #855
• Update kerchunk comparison in FAQ by @TomNicholas in #856

Full Changelog: v2.3.0...v2.4.0

v2.3.0

This release improves support for hierarchical datasets via open_virtual_datatree and makes the consolidated=False kwargs unnecessary with newer Zarr releases via an improvement to ManifestStore.

What's Changed

• Add supports_consolidated_metadata to ManifestStore by @jsignell in #809
• Add top-level open_virtual_datatree by @maxrjones in #838
• Fix broken cross-references by @maxrjones in #846
• Fix typing errors by @maxrjones in #847
• Remove wrapper for kerchunk's tiff parsing by @maxrjones in #849

Full Changelog: v2.2.1...v2.3.0

v2.2.1

Small patch release with a couple bug fixes.

What's Changed

• Allow storing scalar arrays under 'c' key by @maxrjones in #836
• Improve ManifestStore.list_dir for arrays and nested groups by @maxrjones in #837

Full Changelog: v2.2.0...v2.2.1

v2.2.0

Release includes a new Zarr V2 parser, integration with Virtual TIFF for TIFF parsing, and support for parsing HDF data with nested groups, along with bug fixes and documentation improvements.

What's Changed

• Add back supports_partial_writes property, returning False by @maxrjones in #828
• Add link to ESIP talk by @maxrjones in #819
• Add virtual tiff as an optional dependency by @maxrjones in #810
• Fix examples' links by @malmans2 in #795
• Fix typing in ManifestStore by @maxrjones in #820
• Parse zarr v2 by @neilSchroeder in #822
• Remove ZARR_DEFAULT_FILL_VALUE lookup by @norlandrhagen in #812
• Raise informative error on Zarr V2 parsing with Zarr-Python<3.1.3 by @maxrjones in #829
• Return None for Zarr V2/consolidated metadata requests by @maxrjones in #827
• Revert "Remove unnecessary dtype conversion in icechunk writer" by @TomNicholas in #805
• Support nested group in ManifestStore + HDFParser by @ilan-gold in #790

New Contributors

• @malmans2 made their first contribution in #795
• @neilSchroeder made their first contribution in #822
• @ilan-gold made their first contribution in #790

Full Changelog: v2.1.2...v2.2.0

v2.1.2

Patch release with minor bug fixes for the DMRPParser and Icechunk writing behavior.

What's Changed

• New empty release notes by @TomNicholas in #781
• Minor improvements to dmrpp parser by @Mikejmnez in #757
• account for endian being possibly None by @keewis in #787
• Do not validate references for "empty" chunks. by @sharkinsspatial in #791
• Remove unnecessary dtype conversion in icechunk writer by @popododo0720 in #760
• Polish release notes for v2.1.2 by @maxrjones in #793

New Contributors

• @Mikejmnez made their first contribution in #757
• @popododo0720 made their first contribution in #760

Full Changelog: v2.1.1...v2.1.2

v2.1.1

Extremely minor release to ensure compatibility with the soon-to-be released version of xarray (v2025.08.0).

What's Changed

• Empty release notes by @TomNicholas in #778
• Adjust for small change in signature of private xarray function _nested_combine by @TomNicholas in #779
• Eelease notes for v2.1.1 by @TomNicholas in #780

Full Changelog: v2.1.0...v2.1.1

v2.1.0

This release fixes a number of important bugs that could silently lead to referenced data being read back incorrectly.
In particular, note that writing virtual chunks to Icechunk now requires that all virtual chunk containers are set correctly by default.
It also unpins our dependency on xarray, so that VirtualiZarr is compatible with the latest released version of Xarray.
Please upgrade!

What's Changed

• Add template for empty release notes in release guide by @TomNicholas in #755
• Add empty release notes section by @TomNicholas in #754
• Adds Mkdocs alternate tabs to show multiple store configurations on Usage.md by @norlandrhagen in #741
• [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci[bot] in #756
• Include serializer from ManifestStore in Icechunk virtual references by @maxrjones in #766
• Raise instead of silently dropping non-zarr custom codec by @TomNicholas in #775
• Raise if missing virtual chunk containers by @TomNicholas in #774
• Unpin xarray by @maxrjones in #765
• Extract endianness from Bytes codec in V2 metadata conversion by @maxrjones in #769
• Note breaking change of enforcing virtual chunk containers are correct by @TomNicholas in #776
• v2.1.0 release notes by @TomNicholas in #777

Full Changelog: v2.0.1...v2.1.0

v2.0.1

Minor release to ensure compatibility with incoming changes to Icechunk.

What's Changed

• Fix indexing with nones by @TomNicholas in #730
• Add empty release notes section by @TomNicholas in #732
• Refactor indexing by @TomNicholas in #734
• Don't write empty chunks to icechunk by @TomNicholas in #745
• Fix test paths to make upstream Icechunk happy by @paraseba in #752
• Document recommendation to use repo.save_config by @TomNicholas in #746
• Enable workflow_dispatch for upstream workflow by @paraseba in #751
• Add comment on skip_signature=True in the s3 store example by @jbusecke in #731
• v2.0.1 release notes by @TomNicholas in #753

New Contributors

• @paraseba made their first contribution in #752

Full Changelog: v2.0.0...v2.0.1

v2.0.0

🎉 v2.0.0 - Second major release of VirtualiZarr 🎉

Highlights

• Implement open_virtual_mfdataset for easy merging/concatenation with parallelization via Dask, Lithops, or a custom subclass of concurrent.futures.Executor
• Add support for custom external parsers
• Add an internal Zarr V3 Parser
• Enable loading data directly via Zarr and/or Xarray without temporary serialization via Icechunk or Kerchunk
• 📖 Revamped documentation

Detailed Release Notes

Read more about the numerous new features, enhancements, bug fixes, and internal improvements in the release notes.

Migration Guide

The migration guide contains detailed information about switching from VirtualiZarr 1.0 to VirtualiZarr 2.0.