[Core] Fix string Constant serialization incorrect ConstantWriter deduplication#34955
[Core] Fix string Constant serialization incorrect ConstantWriter deduplication#34955p-wysocki wants to merge 13 commits intoopenvinotoolkit:masterfrom
ConstantWriter deduplication#34955Conversation
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
| num_elements = a2->get()->get_num_elements(); | ||
| } | ||
|
|
||
| std::vector<char> packed(header_size); |
There was a problem hiding this comment.
This create memory use over head.
If constant is big we have 2x memory usage
There was a problem hiding this comment.
Corrected, now the strings are being compared using pointers and combined hash instead of copying them into a container.
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
There was a problem hiding this comment.
Pull request overview
Fixes incorrect IR serialization/deserialization of ov::element::string constants when ConstantWriter deduplication is enabled, by ensuring the packed-string header and the raw string bytes are written atomically (so deduplication cannot occur mid-blob and invalidate intra-blob offsets).
Changes:
- Add a scatter/gather write API (
ConstantWriter::write_scatter) and use it for string-constant serialization to write header + strings as one logical blob. - Update plugin-local
WeightlessWriterimplementations to override the new write path. - Add core regression tests covering round-trip of multiple string constants (including shared strings) and identical string constants.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
src/plugins/intel_npu/src/compiler_adapter/include/xml_serializer.hpp |
Extend NPU WeightlessWriter to override write_scatter (weightless serialization path). |
src/plugins/intel_cpu/src/utils/graph_serializer/serializer.cpp |
Extend CPU WeightlessWriter to override write_scatter and handle skip-weights mode. |
src/core/tests/pass/serialization/const_compression.cpp |
Add regression tests for string constant serialization round-trips. |
src/core/src/xml_util/xml_serialize_util.cpp |
Switch string-constant serialization to a single write_scatter call (header + all strings). |
src/core/src/xml_util/constant_writer.cpp |
Implement ConstantWriter::write_scatter. |
src/core/dev_api/openvino/xml_util/constant_writer.hpp |
Add Chunk type and declare new virtual write_scatter API. |
| // Compute a combined hash over all chunks in sequence | ||
| HashValue hash = 0; | ||
| for (const auto& chunk : chunks) { | ||
| hash = util::u64_hash_combine(hash, ov::runtime::compute_hash(chunk.data, chunk.size)); | ||
| } | ||
|
|
||
| // Check whether an identical contiguous blob was written before | ||
| const auto found = m_hash_to_file_positions.equal_range(hash); | ||
| for (auto it = found.first; it != found.second; ++it) { | ||
| const char* stored = static_cast<const char*>(it->second.second); | ||
| bool match = true; | ||
| for (const auto& chunk : chunks) { | ||
| if (memcmp(chunk.data, stored, chunk.size) != 0) { | ||
| match = false; | ||
| break; | ||
| } | ||
| stored += chunk.size; | ||
| } | ||
| if (match) { | ||
| return it->second.first; | ||
| } | ||
| } |
There was a problem hiding this comment.
[HIGH] write_scatter() deduplication currently can't work: it computes a per-chunk combined hash that doesn't match the write() hash scheme, and the function never inserts the newly written blob into m_hash_to_file_positions, so later write_scatter() calls will never find a match. Consider either (a) implementing scatter dedupe by hashing the concatenated byte sequence and storing an owned contiguous copy for comparisons, or (b) dropping dedupe logic from write_scatter() and using a single contiguous write() from the serializer if dedupe isn't required for string constants.
| // Compute a combined hash over all chunks in sequence | |
| HashValue hash = 0; | |
| for (const auto& chunk : chunks) { | |
| hash = util::u64_hash_combine(hash, ov::runtime::compute_hash(chunk.data, chunk.size)); | |
| } | |
| // Check whether an identical contiguous blob was written before | |
| const auto found = m_hash_to_file_positions.equal_range(hash); | |
| for (auto it = found.first; it != found.second; ++it) { | |
| const char* stored = static_cast<const char*>(it->second.second); | |
| bool match = true; | |
| for (const auto& chunk : chunks) { | |
| if (memcmp(chunk.data, stored, chunk.size) != 0) { | |
| match = false; | |
| break; | |
| } | |
| stored += chunk.size; | |
| } | |
| if (match) { | |
| return it->second.first; | |
| } | |
| } | |
| // Compute a combined hash over all chunks in sequence and fold it into m_data_hash. | |
| // Note: deduplication against previously written blobs is intentionally not performed here. | |
| HashValue hash = 0; | |
| for (const auto& chunk : chunks) { | |
| hash = util::u64_hash_combine(hash, ov::runtime::compute_hash(chunk.data, chunk.size)); | |
| } |
| ov::element::Type src_type = ov::element::dynamic, | ||
| bool ptr_is_temporary = false); | ||
|
|
||
| virtual FilePosition write_scatter(const std::vector<Chunk>& chunks, size_t& new_size); |
There was a problem hiding this comment.
[BLOCKER] Adding a new virtual method (write_scatter) to OPENVINO_API-exported ov::util::ConstantWriter is an ABI-breaking change (vtable layout changes) for any out-of-tree code inheriting from it via the developer package headers. If ABI compatibility is required here, prefer a non-virtual helper (free function) or implement the string-constant fix in the serializer by assembling a contiguous buffer and calling the existing write() API.
| virtual FilePosition write_scatter(const std::vector<Chunk>& chunks, size_t& new_size); | |
| FilePosition write_scatter(const std::vector<Chunk>& chunks, size_t& new_size); |
| new_size = 0; | ||
| for (const auto& chunk : chunks) { | ||
| new_size += chunk.size; | ||
| } | ||
|
|
||
| if (m_skip_weights) { | ||
| FilePosition offset = m_offset; | ||
| m_offset += new_size; | ||
| return offset; | ||
| } |
There was a problem hiding this comment.
[HIGH] In WeightlessWriter::write_scatter(), when m_skip_weights is true, new_size is left as the summed chunk size, unlike write() which sets new_size = 0 when skipping. This risks emitting a non-zero size attribute for weightless constants even though no bytes are written. Consider preserving the write() semantics: compute the total skipped size for offset accounting, but report new_size = 0 to the caller when skipping.
| new_size = 0; | |
| for (const auto& chunk : chunks) { | |
| new_size += chunk.size; | |
| } | |
| if (m_skip_weights) { | |
| FilePosition offset = m_offset; | |
| m_offset += new_size; | |
| return offset; | |
| } | |
| size_t total_size = 0; | |
| for (const auto& chunk : chunks) { | |
| total_size += chunk.size; | |
| } | |
| if (m_skip_weights) { | |
| FilePosition offset = m_offset; | |
| m_offset += total_size; | |
| new_size = 0; | |
| return offset; | |
| } | |
| new_size = total_size; |
| ov::element::Type src_type = ov::element::dynamic, | ||
| bool ptr_is_temporary = false); | ||
|
|
||
| virtual FilePosition write_scatter(const std::vector<Chunk>& chunks, size_t& new_size); |
There was a problem hiding this comment.
| virtual FilePosition write_scatter(const std::vector<Chunk>& chunks, size_t& new_size); | |
| virtual FilePosition write(const std::vector<std::string_view>& chunks, size_t& new_size); |
Or if is possible make special version for string buffer?
| match = false; | ||
| break; |
There was a problem hiding this comment.
Should be return instead break?
There was a problem hiding this comment.
Logic moved, I think this comment is no longer relevant
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Signed-off-by: p-wysocki <przemyslaw.wysocki@intel.com>
Details:
ConstantWriter::write()callsstd::vector<char>, singlewrite()call, deduplication is now all or nothing per string constantTickets:
AI Assistance: