Meeting Notes

tsc/meetings/2026-03-25.md

@@ -0,0 +1,189 @@
+Minutes from OpenVDB TSC meeting, March 25th, 2026, (EDT)
+
+Attendees: *Jeff* L., *Nick* A., *Dan* B., *Andre* P.
+
+Additional Attendees: N/A
+
+Regrets: *Ken* M.
+
+Agenda:
+
+1) Confirm quorum
+2) Secretary
+3) Website & File Caches
+4) DCO/CLA PR Policy
+5) I/O Rewrite: PR Walkthrough
+6) Next meeting
+
+--------------------
+
+1) Confirm quorum
+
+Quorum is present (four of seven TSC members).
+
+2) Secretary
+
+Secretary was Dan Bailey.
+
+3) Website & File Caches
+
+Dan confirmed that old VDB sample file caches have been migrated to
+GitLab LFS under the website repository; all links updated, files are
+at format version 223. Nick to close related open issues.
+
+A known issue remains: two Doxygen HTML outputs share the same filename
+under different capitalizations (NanoVDB IO.h vs OpenVDB IO.h), causing
+conflicts on case-insensitive filesystems (Windows, and to a lesser
+extent macOS). Website repository changes currently require a Linux
+machine to avoid this problem.
+
+4) DCO/CLA PR Policy
+
+Several open PRs are missing DCO sign-off or a signed CLA, making them
+un-mergeable. Agreed approach:
+
+- Dan to post comments on all affected PRs giving contributors one week
+  to resolve sign-off; PRs that remain unaddressed will be closed with
+  an invitation to reopen once the requirements are met.
+- Dan to add an explicit policy statement to the contribution
+  guidelines so contributors are aware of this policy up front.
+
+5) I/O Rewrite: PR Walkthrough
+
+Dan shared his screen and walked through a series of PRs on the
+io_point_codecs feature branch. Dan noted these have been broken into
+small reviewable pieces; feedback on each is welcome before they are
+merged.
+
+a) Test CMakeLists: wildcard removal
+
+Removed the implicit wildcard appended to test-name filters in CMake so
+that passing "tree" no longer also runs unrelated tests prefixed with
+"tree". Callers can restore the old behavior with an explicit glob
+pattern (e.g., "tree*"). Jeff and Nick approved.
+
+b) Unit test: stderr suppression
+
+Redirected stderr around a section of unit tests that intentionally
+invokes an API path expected to emit a warning, to reduce noise when
+running with GTEST_BRIEF. Jeff raised the concern that silencing output
+may mask future unexpected warnings. Nick noted that in this specific
+case the warning is the correct behavior of the API under test. Jeff
+approved.
+
+c) TreeBase: readTopology / writeTopology made pure virtual
+
+Moved legacy buffer-count read/write (a pre-open-source 32-bit integer
+always valued 1) from TreeBase into the Tree derived class, and made the
+virtual methods pure virtual on TreeBase. ABI is unchanged (vtable slot
+preserved). This is technically a breaking change for anyone subclassing
+TreeBase directly; judged to be extremely rare. Nick to verify there is
+no impact on a streaming VDB implementation at Weta.
+
+d) PointDataIO.H — new header, multi-pass restriction,
+   out-of-core atomic removal
+
+Three changes bundled together:
+
+- New PointDataIO.H header isolates readCompressedValues for
+  PointDataIndex32, removing the dependency on the full PointDataGrid
+  header for I/O code.
+- Multi-pass I/O restricted to PointDataLeaf only via a static_assert
+  type-trait; the generic multi-pass unit test is removed. This
+  simplifies codec implementation for all non-point tree types.
+  Nick suggested renaming the multi-pass tag class to be more
+  explicitly PointData-specific, or moving it into PointDataIO.H, so
+  that the restriction is obvious at the point of use; Dan agreed to
+  investigate.
+- Out-of-core atomic removed from AttributeArray read/write paths in
+  this branch (delayed loading not supported); removal guarded by an
+  ABI_14 macro for eventual full removal in the next ABI version.
+
+e) Large I/O refactoring (Archive / File / Stream)
+
+A significant refactoring pass to consolidate grid I/O:
+
+- All grid-level I/O now flows through two methods — archiveReadGrid
+  and archiveWriteGrid — eliminating several scattered grid creation
+  sites; grid objects are now created in a single location inside
+  readGrid.
+- Pimpl pattern removed from File and Stream; data members inlined.
+  Jeff flagged a potential header-bloat concern; Dan confirmed no new
+  includes were required. The pimpl was likely introduced for
+  delayed-loading and Boost isolation reasons; no longer warranted.
+  Nick noted symbol visibility as an additional historical motivation
+  but agreed removal is acceptable here.
+- GridDescriptor::read() deprecated (not removed); it created a grid
+  object that callers never used. readHeader and readStreamPosition
+  introduced as explicit replacements. Nick confirmed that Weta uses
+  the stream-level API, not the GridDescriptor API directly.
+- readGridPartial read-topology flag removed (always called with
+  false in practice); partial-read folded into readGrid as a boolean
+  argument.
+- World-space bounding box conversion to index-space deferred until
+  the last moment inside readGrid, eliminating the intermediate local
+  readBuffers struct and simplifying the call path.
+
+Nick and Jeff approved with the suggestions above incorporated.
+
+f) ReadOptions / WriteOptions API
+
+ReadOptions and WriteOptions objects are now passed uniformly to all I/O
+API entry points.
+
+- ReadOptions currently holds: world-space clip bounding box and
+  topology-only read mode (matching the two options previously passed
+  as separate arguments).
+- WriteOptions currently empty; reserved for future extension.
+- All public API methods default-construct these objects; internal
+  methods do not.
+
+Feedback:
+
+- Jeff: C++20 aggregate (named) brace initialization would give
+  keyword-argument ergonomics; Dan to verify and document once VDB
+  requires C++20.
+- Nick: virtual destructor on the structs should be removed if nothing
+  derives from them; Dan to confirm ReadOptions is not subclassed
+  (per-codec options use a separate typed-options map).
+- Nick / Andre: shared_ptr typedefs should be removed if unused; prefer
+  unique_ptr or no typedef. Shared-ptr usage likely a legacy of the
+  pimpl pattern now removed; agreed to clean up.
+
+g) Codec framework — IOCodec, TopologyCodec, BoolCodec, registry
+
+New framework for pluggable grid I/O codecs:
+
+- IOCodec base class exposes: readTopology, writeTopology, readBuffers,
+  writeBuffers, and createData (pure virtual).
+- TopologyCodec intermediate class implements standard depth-first tree
+  topology read/write; multiple variants are planned for different
+  compression schemes. Jeff clarified the hierarchy: codecs that need
+  a specific topology compression derive from the appropriate
+  TopologyCodec variant and only implement buffer methods; fully custom
+  codecs (e.g., a neural codec) derive directly from IOCodec.
+- BoolCodec is the first concrete codec, deriving from TopologyCodec
+  and overriding the buffer methods.
+- Codec registry mirrors the GridRegistry and MetaMap patterns; the
+  codec is looked up by grid-type string inside archiveReadGrid and
+  falls back to existing behavior when no codec is registered.
+- createData hook on the codec allows it to return a different grid type
+  than the one it is registered under (e.g., reading a float grid as a
+  half grid).
+- Per-codec options: ReadOptions contains a std::map<string,
+  TypedOptions> for type-specific settings (e.g., filtering VDB Points
+  attributes by name). Jeff suggested using each codec's globally-unique
+  identifier string as a key prefix to avoid collisions; Dan agreed and
+  will document the convention.
+- Error / warning reporting: currently throws on the first error. Jeff
+  requested a mechanism for non-fatal warnings (e.g., a partial read
+  silently loading the full grid because the codec does not support
+  clipping); Dan to design a message list with string + severity-level
+  fields, keeping hard errors as exceptions.
+
+Dan noted the branch is still in progress; once these foundational PRs
+are merged, additional codecs (scalar, point-data, etc.) will follow.
+
+6) Next meeting
+
+April 8th, 2026, usual time.

tsc/meetings/2026-04-08.md

@@ -0,0 +1,157 @@
+Minutes from OpenVDB TSC meeting, April 8th, 2026, (PDT)
+
+Attendees: *Dan* B., *Nick* A., *Jeff* L., *Andre* P., *Jonathan* S.
+
+Additional Attendees: N/A
+
+Regrets: *Ken* M.
+
+Agenda:
+
+1) Confirm quorum
+2) Secretary
+3) SIMD Support via VCL (MR review)
+4) I/O Rewrite Updates
+5) Custom Grid Type Discussion
+6) Next meeting
+
+--------------------
+
+1) Confirm quorum
+
+Quorum is present.
+
+2) Secretary
+
+Secretary is Dan Bailey.
+
+3) SIMD Support via VCL (MR review)
+
+Nick presented a new MR introducing SIMD support in OpenVDB using Agner Fog's
+Vector Class Library (VCL) as an optional, header-only dependency bundled in
+the `ext/` folder.
+
+Background: VDB currently avoids explicit SIMD and relies on compiler built-ins
+(e.g., PopCount, count-leading-zeros) via x86 intrinsics or platform-specific
+workarounds, with some confusion around when software fallbacks are needed. This
+MR formalises x86 SIMD usage and lays groundwork for broader vectorisation.
+
+Library choice:
+- VCL (Agner Fog): Apache 2, fully header-only in the configuration used.
+  Supports SSE2 through AVX512 with emulation for sizes not natively supported by
+  the targeted instruction set. Previously blocked by license incompatibility;
+  now resolved since OpenVDB is itself Apache 2.
+- Google Highway was considered but rejected as it is not header-only (requires
+  runtime linking) and introduces a heavier external dependency.
+
+MR contents:
+- VCL added as an optional `ext/` subdirectory, installed with OpenVDB.
+- New `SIMD.h` header wrapping VCL into a custom OpenVDB VCL namespace to allow
+  mixing other VCL versions downstream. Provides an `x86 instruction-set` macro
+  (integer, following VCL values) written into `version.h` at pre-process time.
+- Generic API in `SIMD.h` covering VCL container types, `openvdb::math` tuple
+  types, and plain arithmetic scalars, enabling type-agnostic algorithm
+  implementations via a single templated function.
+- Type aliases (e.g., `Vec4F`, `Vec8S`) that resolve to either a VCL SIMD
+  container or a math tuple depending on whether VCL is enabled.
+- A `SIMD_NATIVE_TYPE` macro returning the largest SIMD container type for a
+  given element type and the targeted instruction set.
+- First vectorised tool: rasterised SDF spheres. Achieves a 2× throughput on
+  doubles with no observable floating-point differences across architectures or
+  optimisation levels. Uses `getBatchSize()` and a new `rasterizePoints` range
+  overload to batch-process points.
+
+CMake and detection:
+- New `USE_VCL` CMake option, off by default.
+- Detection is compile-time only, driven by compiler flags (`-mavx`, `-msse4.2`,
+  etc.), not the host machine's capabilities. Compatible with CMake's native
+  cross-compilation toolchain.
+- The two legacy defines (`OPENVDB_USE_AVX`, `OPENVDB_USE_SSE42`) are moved from
+  build-time emission into `version.h`, so downstream header consumers now also
+  see the configured instruction set. Nick noted these defines are inaccurate
+  (they reflect CMake options, not whether the compiler flag was actually passed)
+  and would like to remove them. Jeff suggested simply relying on VCL's
+  detection from compiler flags, which package managers already set correctly.
+- Dan suggested emitting a CMake error if the old defines are set without the
+  corresponding compiler flags, to guide migration; Nick acknowledged this is
+  tricky to implement reliably.
+
+Review feedback and agreed follow-ups:
+- Namespace/location: move the SIMD API from `openvdb::util::simd` to
+  `openvdb::simd`, in a dedicated `simd/` folder. Jeff agreed this is
+  appropriate given its fundamental nature; using `openvdb::simd::` also avoids
+  extra `using` declarations in code already in `namespace openvdb`.
+- Readability: Jeff raised concern about verbose explicit function calls (e.g.,
+  `simd::square(simd::sub(cx, px))`) vs operator syntax. Nick acknowledged the
+  verbosity and noted explicit casting is intentional given that type promotions
+  are now surfaced. He will investigate adding operator overloads in a follow-on
+  PR; this is not a blocker for merging the current MR. Jeff noted binary
+  operators make SIMD code significantly easier to read.
+- Template instantiation: Jeff suggested collapsing large type-trait
+  instantiation lists using macros for clarity. Nick agreed to do this.
+- Testing: Dan requested a small test (potentially static assertions) that
+  validates the correct instruction path is active for a given configuration, to
+  catch accidental mis-wiring. Nick will add this.
+- CI: Nick will add a CI configuration with `USE_VCL` enabled.
+- Default on/off: consensus to keep `USE_VCL` off by default now. Plan to
+  evaluate enabling it by default for VDB 14. Jeff suggested turning it on
+  experimentally in CI around November.
+- Ken's email regarding NanoVDB SIMD was noted; Nick expects no conflict since
+  NanoVDB already copies the OpenVDB math library and the new SIMD headers are
+  independent.
+- Andre indicated support for the VCL approach.
+- C++ standard SIMD (`std::simd`, expected in C++26): noted as a long-term
+  migration target; estimated 10–12 years before VFX platform reaches C++26.
+  The current namespace wrapping is intended to ease that future transition.
+
+4) I/O Rewrite Updates
+
+Dan raised two items related to his ongoing I/O rewrite work.
+
+a) File format backward-compatibility fix:
+  An off-by-one error was introduced when removing legacy I/O modes. The current
+  code supports files from node-mask compression (format version 223, VDB 3,
+  ~2013) onwards, but inadvertently dropped support for version 222. The
+  intention was to support from VDB 1.1 (the first Houdini release) onwards.
+  Dan will restore the missing branch to re-enable reading version 222 files.
+  Jeff confirmed this was the agreed scope. A small number of regression-test
+  sample files predated version 223.
+
+b) ReadDiagnostics API design:
+  Dan described the current implementation: `ReadDiagnostics` is a nullable
+  pointer passed into codecs; codecs check for null before populating it.
+  Callers opt in by calling `enableReadDiagnostics()` on the archive, which
+  allocates the object.
+
+  Nick suggested an alternative: `ReadDiagnostics` always exists as a value
+  object, with `addWarning()` being a no-op when not enabled, removing the
+  risk of null-pointer mistakes in codec implementations. Jeff agreed and
+  proposed adding an `enabled` flag to the object so that codecs can also
+  gate expensive diagnostic-collection logic cheaply. Dan will adopt this
+  approach: always-present object with an `enabled` flag, with `addWarning()`
+  being a no-op when disabled.
+
+c) Feature branch review process:
+  Dan has ~4–5 stacked MRs targeting an I/O feature branch rather than master,
+  with each MR targeting the previous branch for reviewable diffs. He asked for
+  guidance on review cadence. Jeff noted that the team needs to merge these
+  incrementally or the stack will become unwieldy. The refactoring MR (large API
+  changes to unify read/write paths and simplify codec integration) is the
+  critical one that unblocks subsequent PRs. Team agreed to prioritise reviewing
+  these promptly.
+
+5) Custom Grid Type Discussion
+
+Dan asked about a Weta-internal streaming grid type that Nick had mentioned, to
+understand how a non-`TreeBase`-derived grid handles I/O — useful context for
+the I/O rewrite's generality.
+
+Nick: the implementation is a full reimplementation of the tree hierarchy (grid,
+tree, root, internal, and leaf nodes, and leaf buffer), with a unique grid type
+deriving from `GridBase` (not `TreeBase`). It has custom K-I/O. Nick is working
+on open-sourcing it, but Dan should not wait on it and should proceed with his
+I/O changes independently.
+
+6) Next meeting
+
+April 22nd, 2026, usual time.