Add composable store middleware system

[manzt]

Introduces a composable store middleware system for wrapping stores with additional behavior (caching, batching, consolidated metadata, etc.) via Proxy delegation.

Core APIs:

• zarr.defineStoreMiddleware(factory) — define a middleware that intercepts store methods. The factory receives an AsyncReadable store and options, returns overrides and extensions. Unoverridden methods are automatically delegated via Proxy.
• zarr.defineStoreMiddleware.generic<OptsLambda>()(factory) — variant for middleware whose options depend on the store's request options type (uses GenericOptions for higher-kinded type encoding).
• zarr.storeFrom(store, ...middleware) — compose middleware in a pipeline. Each step is an arrow function that receives the store with full type inference.

import * as zarr from "zarrita";

let store = await zarr.storeFrom(
  new zarr.FetchStore("https://..."),
  zarr.withConsolidation,
  (s) => zarr.withRangeBatching(s, {
    mergeOptions: (batch) => batch[0],
    //             ^? ReadonlyArray<RequestInit | undefined>
  }),
);

store.contents(); // from consolidation
store.stats;      // from batching

Built-in middleware:

• zarr.withConsolidation — loads consolidated metadata (v2/v3), intercepts get for cached metadata lookups, adds contents() for listing
• zarr.withRangeBatching — batches concurrent getRange() calls within a microtask tick, coalesces adjacent ranges, LRU caching

[changeset-bot]

🦋 Changeset detected

Latest commit: a0b6bfe

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
zarrita	Minor
@zarrita/ndarray	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[kylebarron]

In case it's any relevant related work, I've been working on a sort of "object store generic store" interface in obspec, but still working on how to have a generic interface on that.

I've been trying to follow the object-store model of "wrappers" like this

[manzt]

Here is a bigger example:

import * as zarr from "zarrita";

// Simple middleware — adds getRange fallback for stores that lack it
const withEnsureGetRange = zarr.defineStoreMiddleware((store) => ({
  async getRange(key, range, opts) {
    if (store.getRange) return store.getRange(key, range, opts);
    let data = await store.get(key, opts);
    if (!data) return undefined;
    if ("suffixLength" in range) {
      return data.slice(data.length - range.suffixLength);
    }
    return data.slice(range.offset, range.offset + range.length);
  },
}));

// Generic middleware — opts are inferred from the store's options type
interface LoggingOptions<O = unknown> {
  format?: (key: string, options: O | undefined) => string;
}

interface LoggingOptsFor extends zarr.GenericOptions {
  readonly options: LoggingOptions<this["_O"]>;
}

const withLogging = zarr.defineStoreMiddleware.generic<LoggingOptsFor>()(
  (store, opts: LoggingOptions = {}) => {
    let fmt = opts.format ?? ((key) => `GET ${key}`);
    return {
      async get(key, options) {
        console.log(fmt(key, options));
        return store.get(key, options);
      },
    };
  },
);

// Compose in a pipeline — arrow functions give full type inference
let store = await zarr.storeFrom(
  new zarr.FetchStore("https://example.com/data.zarr"),
  withEnsureGetRange,
  (s) =>
    withLogging(s, {
      format: (key, opts) => `${key} [signal=${opts?.signal?.aborted}]`,
      //                                      ^? RequestInit | undefined
    }),
  zarr.withConsolidation,
  (s) => zarr.withRangeBatching(s, { mergeOptions: (batch) => batch[0] }),
);

store.contents(); // from consolidation
store.stats; // from batching