CHANGELOG.md

Changelog

All notable changes to MRRC will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[Unreleased]

Added

Changed

Fixed

Performance

Documentation

[0.9.2] - 2026-07-27

Security

• Bump quick-xml to 0.41.0, fixing the RUSTSEC-2026-0194/0195 DoS advisories on the MARCXML read/write path. The BIBFRAME/RDF path is not covered: oxrdfxml pins quick-xml ^0.37 and still resolves to the affected 0.37.5. Since bibframe is a default feature (and is always enabled in the Python wheel), a default build still contains the affected code on the RDF serialization path. Both advisories are DoS-class.
• Bump crossbeam-epoch to 0.9.20, fixing the RUSTSEC-2026-0204 invalid-pointer-dereference advisory (reached transitively through rayon).
• Gate PyPI publishing behind a required-reviewer GitHub environment, so a release cannot upload without maintainer approval.
• Harden the CI and release workflows: pin all third-party actions to commit SHAs, disable git-credential persistence in checkouts, scope job token permissions to least privilege, and add a zizmor workflow-security scan to CI.

Dependencies

• Bump pytest from 9.1.0 to 9.1.1
• Bump ruff from 0.15.17 to 0.15.21
• Bump pymdown-extensions from 10.21.3 to 11.0.1
• Bump foldhash from 0.1.5 to 0.2.0
• Bump CodSpeedHQ/action from 4.17.5 to 4.18.5
• Bump softprops/action-gh-release from 3.0.0 to 3.0.2
• Bump actions/checkout from 6.0.3 to 7.0.0
• Bump actions/cache from 5 to 6.1.0
• Bump pyright from 1.1.410 to 1.1.411
• Bump syrupy from 5.3.2 to 5.5.3
• Bump regex from 1.12.4 to 1.13.0
• Bump memchr from 2.8.2 to 2.8.3
• Bump crossbeam-channel from 0.5.15 to 0.5.16
• Bump mypy from 2.1.0 to 2.3.0
• Bump pymarc from 5.3.1 to 5.4.0
• Bump astral-sh/setup-uv from 8.2.0 to 8.3.2
• Bump actions/setup-python from 6.2.0 to 6.3.0
• Bump docker/setup-qemu-action from 4.1.0 to 4.2.0
• Bump anyhow from 1.0.102 to 1.0.103 (dev-dependency, used by examples only; picks up the RUSTSEC-2026-0190 Error::downcast_mut() unsoundness fix, which does not affect the published library or wheel)

[0.9.1] - 2026-06-28

Added

• New cargo-fuzz targets covering the in-memory byte-source parse entry points (parse_record_from_bytes / parse_record_from_shared_bytes) and the three ISO 2709 writers (write_arbitrary_records, over arbitrary records), run nightly alongside the existing binary-MARC targets.

Changed

• ProducerConsumerPipeline sends parsed records to its consumer one batch per file chunk instead of one record at a time. Its channel_capacity argument now counts batches (each one parsed file chunk) rather than records, and defaults to 4; callers that set it explicitly should adjust.

Fixed

• The binary writers now reject a single field whose serialized length exceeds 9999 bytes instead of emitting a corrupt directory. The ISO 2709 directory entry stores field length in a fixed 4-digit field, so an over-long field previously produced a malformed 13-byte entry; it now returns a WriterError naming the offending tag. Affects the bibliographic, authority, and holdings writers.
• parse_batch_parallel (Rust and the Python extension) now returns a MarcError instead of panicking when given record boundaries whose offset + length exceeds the buffer or overflows usize. The function takes caller-supplied boundaries; values from RecordBoundaryScanner are always in range, so this only affects callers passing boundaries directly.

Performance

• Binary ISO 2709 writing is roughly 2.8–3.2× faster: the bibliographic MarcWriter now serializes fields into reused buffers instead of allocating a temporary Vec per field and per record, dropping per-record allocations from ~68 to ~1. Reading is modestly faster (~10%) from removing a redundant per-field tag-string clone in Record::add_field.
• Reading binary MARC is a few percent faster (~5–6% measured): the tag-keyed field maps now hash with foldhash instead of the standard library's SipHash, dropping cryptographic-hash overhead that was unnecessary for short, trusted MARC tags. Record output and field ordering are unchanged.
• Constructing a reader from a Python bytes object no longer copies the whole buffer up front: the reader borrows the immutable bytes and slices records out of it, avoiding a full-dataset copy at construction for large in-memory inputs. bytearray input is unchanged (still snapshotted to an owned buffer).

[0.9.0] - 2026-06-26

Breaking

• Python — record.leader is now a property, not a method (use record.leader, not record.leader()). Leader constructs from a 24-character string or as Leader() with properties assigned; the old field keyword arguments (Leader(record_type=..., ...)) are removed.
• Python — Record.get_fields() returns control fields in record order (was fixed ascending-tag order).
• Rust — MarcError and the PipelineError, RecoveryMode, ValidationLevel, EncodingAnalysis, RdfFormat, and IndicatorValidation enums are #[non_exhaustive]: exhaustive matches need a wildcard arm, and external construction goes through the new MarcError constructors.
• Rust — from_path constructors now return MarcReader<BufReader<File>> (was MarcReader<File>).
• Rust — recovery::try_recover_record is removed; truncated-record salvage now runs through the ISO 2709 skeleton walk.
• Build — MSRV is now Rust 1.88; both crates use edition 2024.

Added

• Community files: issue and pull request templates, and a security policy (SECURITY.md) pointing at GitHub private advisories.
• parse_record_from_bytes: parse one complete MARC record from in-memory bytes with no reader I/O and no per-record copies. The Python MARCReader read path now uses it, collapsing the former chain of per-record buffer copies between the source and the parser to a single pymarc-compatibility stash (current_chunk).
• parse_record_from_shared_bytes: parse from a buffer the caller already holds behind an Arc, without taking ownership. The Python MARCReader uses it so the current_chunk stash shares one allocation with the parser instead of cloning, and current_chunk is now read lazily — iterating without inspecting it copies no record bytes into Python.
• Criterion benches for the serialization formats that had no CI perf signal — CSV, MODS, Dublin Core, BIBFRAME (Turtle), and MARC-in-JSON (both directions for MODS and MARC-in-JSON) — plus a single-thread parser-pool bench that tracks the producer-consumer pipeline's per-record cost deterministically under simulation.
• The pymarc parity oracle now executes in CI and check.sh instead of silently skipping: a new oracle extra pins pymarc in uv.lock (Dependabot adjudicates behavior changes on bump PRs), and the oracle extends beyond iteration shape to value-level comparisons — title, format_field(), value(), and as_marc() byte-equality over the 1k corpus.
• MarcError::metadata() returns an ErrorMetadata snapshot of every structured field an error carries. The per-field accessors, Display/detailed() rendering, JSON output, and the Python exception mapping all read from this single per-variant table now; rendered and serialized output is unchanged.
• MarcError (enum and variants) and the grower enums (PipelineError, RecoveryMode, ValidationLevel, EncodingAnalysis, RdfFormat, IndicatorValidation) are now #[non_exhaustive], so adding a variant or field is no longer a breaking change. Downstream matches need a wildcard arm; MarcError construction outside the crate goes through new public constructors (invalid_field, truncated_record, record_length_invalid, fatal_reader_error) and with_* positional setters.
• copy.deepcopy() support for Record, Field, and Leader: deep copies are fully independent of the original (and of any record), while copy.copy() stays shallow, matching pymarc. Deep-copying a live field handle yields a detached snapshot of its current data.
• A reproducible three-way benchmark (scripts/benchmark_comparison.py, with examples/benchmark_native.rs for the native column) and published figures (docs/benchmarks/): the Python wrapper reads ~7× pymarc per record, ~30× on the parallel bulk path, ~1.6× extract, and ~6.5× roundtrip, against a native-Rust ceiling of ~9×/~35× on the reference host. Replaces the prior "needs re-measurement" hedge.

Changed

• The parallel parse helpers (parse_batch_parallel / parse_batch_parallel_limited) and the blocking ProducerConsumerPipeline reads now release the GIL during the parallel parse and the channel wait, so other Python threads keep running instead of being blocked for the duration. (The parallel helpers now take ownership of the buffer, required for sound GIL release.)
• Record.get_fields() with no arguments is faster: it made one PyO3 call per control tag (nine) plus one per data field, and now fetches all control fields in a single call. Control fields come back in record order rather than fixed ascending-tag order (a difference only for records whose control fields are stored out of order; record order matches pymarc).
• Iterating a MARCReader is faster: wrapping each parsed record no longer builds a throwaway inner _Record and two _Leader objects only to discard them, lifting per-record read throughput (path and bytes inputs alike). Output is unchanged.
• Writing ISO 2709 records (MARCWriter and the authority/holdings writers) allocates less: each directory entry's length and start-position digits are written straight into the output buffer instead of through a per-field format!.
• MARCXML deserialization and 880-linkage parsing no longer recompile their regexes on every call (hoisted to LazyLock statics). Parsing MARCXML records one at a time — e.g. parse_xml_to_array in a loop — is substantially faster; batch/whole-document parsing already amortized the cost and is unchanged.
• Reading from a Python file-like object (MARCReader(open(path, "rb")), BytesIO, etc.) now reads the source in 256 KiB chunks and slices records out in Rust, instead of two file.read() calls plus a getattr("read") per record. The read method is bound once and each chunk is borrowed via PyBytes rather than copied. Output is unchanged; file-path and bytes inputs were already chunked and are unaffected.
• Cargo.lock is now committed, so CI, wheel builds, and local checkouts resolve the same dependency versions; Dependabot manages version bumps as reviewable diffs.
• Removed unused dependencies (bytes, nom, encoding_rs, csv, bindings-crate tempfile) and demoted example-only anyhow/flate2 to dev-dependencies, shrinking the published crate's dependency tree. check.sh now runs cargo machete to keep it so.
• The version is now declared once, in [workspace.package]: both crates inherit it and pyproject.toml reads it via dynamic = ["version"]. Shared dependencies and rust-version are also workspace-inherited, so they cannot drift between crates.
• BIBFRAME/RDF support is now behind the default-on bibframe cargo feature. Default builds and the Python wheel are unchanged; --no-default-features drops the oxrdf/oxrdfio dependency tree for MARC-only users.
• Record parsing no longer copies every record's bytes into the error-diagnostics buffer: the parse buffer is shared by refcount instead, deleting a per-record alloc+memcpy that every read path paid so that the under-1% of records that error could render hex dumps. Error bytes_near output is unchanged.
• Tuned the release profile: fat LTO, codegen-units = 1, and debuginfo stripping. Wheels get ~2-3% higher read throughput and a 21% smaller extension binary.
• The Python leader surface now matches pymarc: record.leader is a property (was a method) and assignable from a Leader or 24-character string; Leader renders and compares as its 24-character MARC 21 string (str(), len(), == with str, repr) and constructs from one (Leader('00136nam a2200061 4500')). The oracle now compares leaders against pymarc value-for-value. Rust Leader gains a Display impl.
• Migrated both crates to Rust edition 2024 and raised the MSRV from 1.87 to 1.88 (let-chains require 1.88). The workspace now uses resolver v3, so dependency resolution respects rust-version instead of breaking the MSRV build on upgrades.
• Releases now publish a source distribution and macOS universal2 wheels, so Intel Macs and platforms outside the wheel matrix can install mrrc (previously "no matching distribution"). PyPI metadata completed: real author, PEP 639 license expression, Typing :: Typed classifier, and project URLs (homepage, docs, changelog, issues).
• File-path readers now buffer their reads (64 KiB): Python MARCReader(path) (and the authority/holdings readers) and Rust from_path previously issued two-plus read(2) syscalls per record. Rust from_path constructors now return MarcReader<BufReader<File>> instead of MarcReader<File>.
• Updated pyo3 to 0.29, resolving RUSTSEC-2026-0176 (out-of-bounds read in nth/ nth_back on PyList/PyTuple iterators in pyo3 ≤0.28).
• Updated quick-xml to 0.40. MARCXML text and attribute decoding now applies XML 1.0 end-of-line and attribute-value normalization explicitly (previously the XML 1.1 EOL set, which additionally folded NEL/LSEP — those are no longer normalized, matching MARCXML's XML 1.0 reality).
• Corrected the declared Rust MSRV from 1.71 to 1.87 — the floor the dependency tree already required, so 1.71 never actually built; a new CI job now verifies the workspace compiles on the declared MSRV so the claim cannot drift again. The accurate MSRV unlocked clippy-driven cleanups: the MARC-8 tables now use std::sync::LazyLock (dropping the lazy_static dependency) and I/O error construction uses std::io::Error::other.
• Truncated records no longer allocate or zero-pad a buffer of the leader's claimed length: the reader grows the record buffer as bytes arrive (8 KiB steps) and lenient salvage parses the short body directly, so allocation is bounded by actual input size instead of a claimed maximum-length record on a small stub.

Removed

• Deleted four dead manual-profiling bench harnesses (profiling_harness, detailed_profiling, rayon_profiling, rayon_file_io_profiling). They produced no CI or CodSpeed signal, referenced documents that no longer exist, and one wrote scratch files to /tmp. The bench target now contains only criterion benches.
• The placeholder mrrc binary (src/main.rs, a banner print) and the orphaned tests/create_sample_data.py (fixtures come from scripts/generate_benchmark_fixtures.py).
• The docs.rs front page no longer embeds the README (whose CI badges and repo-relative links 404 off GitHub); it now renders the curated crate-level docs instead.
• recovery::try_recover_record: truncated-record salvage now runs through the shared ISO 2709 skeleton's clamped directory walk, which slices fields at their data-area offsets — intact fields ahead of the truncation point are now recovered instead of dropped. Non-digit directory bytes on the salvage walk keep their InvalidField (E106) shape but now also count against the reader's recovered-error cap.

Fixed

• Record is now iterable: for field in record yields each field as a wrapped mrrc.Field (control and data, in record order), matching pymarc. It previously raised TypeError.
• Record.remove_field_at() now returns the mrrc.Field wrapper (with __getitem__ and the other pymarc conveniences) instead of the bare _mrrc.Field extension type.
• copy.copy() of a Record no longer raises RecursionError: the wrapper's attribute delegation now stops cleanly when the inner record is absent during copy reconstruction.
• Corrected Python documentation errors that broke copy-pasted code: the README and mrrc.read() examples called record.title() (it is a property); docs showed imports and reader constructors that do not exist in Python (MarcError, MARCReader.from_path(), .with_source()); the API reference stated the wrong recovery_mode default ("strict"; it is "permissive") and omitted the validation_level and max_errors keyword arguments.
• Corrected documentation drift: the Rust API reference showed parse_batch_parallel with the wrong arity and a stale 5-variant MarcError table; parse_batch_parallel / parse_batch_parallel_limited were missing from the Python API reference; the release procedure understated the wheel count and used retired issue-tracker commands.

Dependencies

• Bump ruff from 0.15.16 to 0.15.17
• Bump syrupy from 5.3.1 to 5.3.2
• Bump pytest from 9.0.3 to 9.1.0
• Bump CodSpeedHQ/action from 4.17.0 to 4.17.5

[0.8.2] - 2026-06-09

Added

• mrrc.StaleFieldError (subclass of MrrcException), raised when a field handle is used after fields were removed from its record.
• Coverage-guided fuzz targets for the JSON (parse_json, parse_marcjson), MARCXML (parse_marcxml), MODS (parse_mods), and MARC-8 (decode_marc8) read paths, run nightly alongside the existing binary-MARC targets.
• CI and .cargo/check.sh now run stubtest to verify mrrc/_mrrc.pyi matches the compiled extension, preventing type-stub drift.
• A documentation-build check runs on pull requests that touch docs/, mkdocs.yml, or the Python sources the API reference is generated from, so a broken docs build is caught before merge instead of only on the main deploy. The docs build now uses mkdocs build --strict everywhere (PR check, deploy, and .cargo/check.sh), turning broken links and unresolved mkdocstrings references into errors.
• A process-label lint (in .cargo/check.sh and CI) fails the build when source, tests, or non-CHANGELOG docs embed implementation-process labels (bead IDs, PR/issue numbers, or release-tagged claims like "since 0.8.1"), which belong in git history and the CHANGELOG instead.

Changed

• The Python API reference is now generated from source with mkdocstrings instead of hand-maintained signature tables, so it cannot drift from the code. The page also gained reference sections for the query DSL (FieldQuery, TagRangeQuery, SubfieldPatternQuery, SubfieldValueQuery) and the parallel-processing classes.
• Documentation fixes: the BIBFRAME-to-MARC guide example now uses the real RdfGraph.parse(...) API (it previously referenced a nonexistent BibframeGraph.from_turtle), and docstring examples are fenced so they render as code blocks rather than leaking headings into the reference.
• Reader option arguments (recovery_mode, validation_level, max_errors) are now keyword-only on MARCReader, AuthorityMARCReader, and HoldingsMARCReader, matching the documented signatures; pass them by name. The record source remains the first positional argument.
• The mrrc/_mrrc.pyi type stubs now match the compiled extension exactly: missing members were added and entries absent at runtime were removed, so stub-driven type checking reflects the real API.
• CI PR workflows add cancel-in-progress concurrency groups and per-job timeouts, skip the heavy wheel and benchmark jobs on Dependabot pull requests (lint, tests, and semver still run), and share one rust-cache configuration across the Rust compile jobs.
• Documentation now states explicitly that CSV and Dublin Core are write-only (export) formats: both are lossy projections of a MARC record, so MRRC emits them but does not parse them back into MARC.
• Documentation no longer presents specific throughput figures or pymarc multipliers as current measurements; early benchmark results are summarized with the caveat that they need re-measurement, and the benchmark docs now describe the measurement infrastructure and the procedure for producing citable numbers.
• The Python CodSpeed CI job now runs in simulation mode (previously walltime), making PR performance-regression detection deterministic on hosted runners. Parallel-throughput benchmarks (Python and Rust) are excluded from CodSpeed, since Valgrind serializes threads; they remain runnable locally via pytest and criterion.
• CI Windows jobs are pinned to windows-2025 (previously windows-latest) ahead of GitHub's 2026-06-15 redirect of that label to a new image.
• Pull requests now build and test a slim wheel matrix (8 jobs) instead of the full 30; the full os × python matrix still runs on push to main and on release tags. The standalone Rust build.yml workflow is removed; its examples-compile step moved into the test workflow, and cross-OS core compilation remains covered by the maturin wheel matrix. .cargo/check.sh gains a --release flag to build the Python extension in release mode for perf-sensitive debugging.
• Continuous integration trims redundant work: the Python pull-request benchmark matrix (pytest-benchmark across five Python versions) is dropped in favor of the CodSpeed regression gate, CHANGELOG linting now runs in CI, and commits that touch only .beads/** no longer trigger the test, benchmark, wheel, and lint workflows.
• .cargo/check.sh now compiles examples/ and benchmarks (cargo bench --no-run) in its full pre-push run, catching breakage that previously surfaced only in CI.

Removed

• The Python Record.fields_by_tag(tag) method. Use the pymarc-standard get_fields(tag), which returns the same live-handle fields for a data tag and also accepts multiple tags and control (00X) tags. The Rust Record::fields_by_tag is unchanged.

Fixed

• Record.remove_field(field) now removes exactly the given field instead of every field sharing its tag, matching pymarc; detached fields match by value and raise ValueError when absent. remove_field/remove_fields with a control tag now remove the control field (previously a silent no-op). fields() now enumerates repeated control tags (006/007) identically to get_fields(). (#248)
• Python in-place field edits now persist to the record, matching pymarc. Fields obtained from a record (record[tag], get_field, get_fields, fields) are live handles: indicator and subfield edits, and control-field .data assignment, write through to the record. (#241)
• Corrected documentation to match the current API. The Rust API-reference "Key Methods" tables now use real method names, distinguish public fields from methods, and show accurate return types; the AuthorityRecord / HoldingsRecord examples use the real ::builder(leader) API. Several Rust and Python tutorial/reference examples were fixed (query DSL types, subfields_by_code, error matching, batch parsing, get_fields for pymarc-style access, record.leader()). Removed documentation for MARC-8 output, which is not supported — MRRC writes UTF-8. Reported by @acdha.

Dependencies

• Bump ruff from 0.15.12 to 0.15.15
• Bump syrupy from 5.1.0 to 5.3.1
• Bump pyright from 1.1.409 to 1.1.410
• Bump astral-sh/setup-uv from 7 to 8.1.0
• Bump CodSpeedHQ/action from 4 to 4.17.0
• Bump actions/checkout from 6 to 6.0.3
• Bump codecov/codecov-action from 6 to 6.0.1

[0.8.1] - 2026-05-29

Added

• get_field(tag) and get_field_or_err(tag) accessors on all three record types (Record, AuthorityRecord, HoldingsRecord), in both Rust and Python. get_field returns the first matching field as Option<&Field> / None; get_field_or_err raises mrrc.FieldNotFound (E105) with field_tag and record_control_number populated. Existing get_fields is unchanged.
• max_errors kwarg on Python MARCReader. Caps the total recovered errors across a lenient / permissive stream; the read after the (N+1)-th raises mrrc.FatalReaderError (E099). None (default) and 0 both disable the cap. Inert in strict mode.
• Documented pymarc 5.3.1 exception-class-name parity in the error-handling reference and the migration guide: the mapping table, the names mrrc deliberately omits, the known hierarchy divergences, and porting recipes.
• MARCReader.current_exception and MARCReader.current_chunk — pymarc-compatible accessors. After each __next__, current_chunk holds the bytes just read and current_exception holds the exception swallowed under permissive=True (or None on a clean read). See the migration guide for the encoding-strictness divergence.
• New validation_level reader kwarg ("structural" default, "strict_marc"), orthogonal to recovery_mode and applied uniformly across all three readers. At strict_marc, indicator bytes (E201), subfield-code bytes (E202), and UTF-8 decoding (E301) enforce MARC 21 byte-level rules; at structural those bytes are accepted as-is and invalid UTF-8 falls back to U+FFFD.
• Per-record diagnostics. record.errors (Rust + Python) carries the typed exceptions for non-fatal defects recovered in lenient / permissive modes (always empty in strict). MARCReader.iter_with_errors() yields (record, errors) tuples, and under permissive=True yields (None, [exception]) so unsalvageable records stay observable. record.errors is on all three readers; iter_with_errors is bibliographic-only.
• Strict-mode parsing now verifies that the byte at the leader's claimed end-of-record position is RECORD_TERMINATOR (0x1D); a different byte fires EndOfRecordNotFound (E006). Previously the byte was unchecked and a malformed record with the wrong terminator parsed silently. Lenient and permissive modes are unchanged — the recovery cap continues to absorb the disagreement via existing directory/field paths.
• validation_level="strict_marc" also runs MARC 21 semantic checks: per-tag indicator rules (e.g. 245 first indicator must be 0 or 1) and leader-byte semantics, firing E201 and E002 respectively (both recoverable in lenient / permissive). IsbnValidator / EncodingValidator remain opt-in helpers; see the validators reference.
• MarcError now implements Clone (Rust), enabling inspection of recovered errors on record.errors after lenient parsing. Clone is lossy for the three variants wrapping foreign causes (IoError, XmlError, JsonError): it preserves the rendered message but drops the non-string inner cause.
• SubfieldPatternQuery now exposes its regex via a pattern getter (Rust SubfieldPatternQuery::pattern()), and its repr() includes the pattern — <SubfieldPatternQuery tag=084 subfield=a pattern="^abc"> — so the most useful field for debugging a query is recoverable from the REPL or logs. The tag and subfield_code getters are now also declared in the type stubs. Thanks to @acdha (#226).

Changed

• Leader errors from the MARCXML, JSON, and marcjson readers now carry record_index (previously stripped) — identifying the failing record in a multi-record collection, or 1 for single-record APIs. Affects E001–E004 from these paths; the ISO 2709 path was already enriched.
• MarcError::IoError (E007) raised mid-record — when the underlying source fails while reading a record's data area — now carries record_index, byte_offset, and source_name, instead of the context-free From<io::Error> fallback that left them None. I/O failures at a record boundary (before a record is in progress) stay context-free by design; Python's OSError surface is unchanged.
• Retired the internal cumulative-budget perf-gate CI workflow added during the error-handling epic — it served its purpose and the cumulative v0.8.0→v0.8.1 hot-path cost stayed negligible. Codspeed continues general perf tracking, and the benchmarks still run locally.
• Python MARCReader / AuthorityMARCReader / HoldingsMARCReader now default to recovery_mode="permissive" (was "strict"), matching the pymarc / marc4j convention — a fresh reader iterates past per-record defects instead of aborting on the first. The Rust core's mrrc::MarcReader keeps Strict. Pass recovery_mode="strict" explicitly for the old behavior; the permissive=True pymarc-compat path is unchanged. See the error-handling guide for the trade-offs.
• Leader-validation errors now fire the field-specific variants their documentation describes: RecordLengthInvalid (E001) for non-digit bytes 0-4 or record_length < 24, BaseAddressInvalid (E003) for non-digit bytes 12-16 or data_base_address < 24, and BaseAddressNotFound (E004) for data_base_address > record_length. Previously all of these collapsed to InvalidLeader (E002) or, for E004, fell through to a misleading InvalidField from directory parsing.
• A non-digit byte in a directory entry's length or start-position field now fires DirectoryInvalid (E101) with field_tag and a precise byte_offset, instead of the misleading InvalidField (E106) it previously forwarded.
• The performance-tuning, migration-from-pymarc, and working-with-large-files guides now point to the Query DSL guide where field filtering is discussed, so readers discover the indicator/range/pattern/subfield matching path. Thanks to @acdha (#234).
• The Rust examples throughout the docs (quickstart, tutorials, reference) now match the real API and compile: field.get_subfield('a') (char), the public tag / indicator1 / indicator2 fields, the FieldQuery::new() builder with record.fields_matching(&query), Leader::from_bytes(...), the record_to_json / record_to_marcxml / record_to_marcjson conversion functions, and use mrrc::RecordHelpers; for record.title(). Many previously referenced methods that don't exist. Reported by @acdha (#233).

Fixed

• Two out-of-bounds slice panics in the lenient/permissive recovery path: salvage attempts no longer crash when a directory entry's start_position lies past the buffer (now bailing out of the salvage branch), and the control-field decode path now guards against zero-length directory entries (where end_position == start_position) before invoking saturating_sub(1) on the slice end. Surfaced by the recovery-mode-consistency fuzz target.
• MarcReader rejects non-ASCII bytes in directory entry tags (firing DirectoryInvalid / E101) instead of lossily substituting U+FFFD and producing records whose tag re-encodes to more than 3 bytes. MarcWriter and the authority/holdings writers also refuse records whose tags aren't 3 ASCII bytes, returning WriterError (E404). Surfaced by the error-classification fuzz target's round-trip assertion.
• TruncatedRecord (E005) now surfaces on record.errors in lenient and permissive modes instead of being silently swallowed (it had been cascading into a misleading E201). Strict mode is unchanged.
• Python mrrc.MARCReader now honors recovery_mode on short body reads: a truncated body no longer raises TruncatedRecord (E005) before the recovery-aware parser runs, so in lenient / permissive it lands on record.errors instead. Strict mode still raises.
• Release workflow now attaches wheel assets to the GitHub Release page automatically. Previously, actions/checkout ran after download-artifact and wiped dist/ before the gh-release step, leaving the release page with notes but zero assets. Steps reordered so checkout runs first.
• MarcWriter and the authority/holdings writers now refuse records whose serialized length or base address exceeds the ISO 2709 5-digit limit (99999 bytes), returning WriterError (E404) with positional context — instead of silently emitting an unparseable leader (or, for holdings, the wrong InvalidField variant).
• mrrc.MARCWriter.write_record now raises the typed mrrc.WriterError (E404) instead of a plain OSError; the binding had been collapsing every writer MarcError into io::Error across the FFI boundary.
• InvalidLeader (E002) errors now carry the full positional context (record_index, byte_offset, record_byte_offset, source_name) the v0.8.0 error work promised; the leader-validation path had been building the variant message-only and discarding every positional field.
• TruncatedRecord (E005) raised across the Python FFI now preserves the same positional context (record_index, byte_offset, record_byte_offset) the Rust core attaches; Python callers previously saw a wrong actual_length and missing stream-position metadata.
• AuthorityMarcReader and HoldingsMarcReader at validation_level="strict_marc" no longer trip InvalidLeader (E002) on leader bytes valid for their own record type. Each reader now applies its own MARC 21 format's leader rules (Authority / Holdings) rather than the Bibliographic allowed-value sets; bibliographic dispatch is unchanged.

Dependencies

• Bump urllib3 from 2.6.3 to 2.7.0
• Bump mypy from 1.20.2 to 2.1.0

[0.8.0] - 2026-04-29

Breaking

• MarcError variants restructured to struct form carrying positional metadata (record_index, byte_offset, record_byte_offset, record_control_number, field_tag, indicator_position, subfield_code, found, expected, source_name). Wrapping variants (IoError, XmlError, JsonError) chain via #[source] so Error::source() walks correctly; Display output changed for every variant. Direct constructors and pattern matches need updating. Migration: match on .code() ("E001"– "E099", "E101"+) instead of variant names — codes are stable across future enum changes.
• MarcError::InvalidRecord removed. Use the new specific variants (DirectoryInvalid, RecordLengthInvalid, …) or InvalidField as the fall-through.
• MarcError::ParseError removed. Same migration as InvalidRecord: the new specific variants and InvalidField cover the cases that ParseError used to wrap. Match on .code() for stable identity.
• recovery::RecoveryContext removed from the public re-exports. No external consumer; the type was an internal helper that accumulated messages and dropped them. Pattern-match on MarcError instead.
• recovery::try_recover_record takes a &ParseContext as a fifth parameter. The previous v0.7 signature constructed RecoveryContext internally; v0.8 callers pass the ParseContext that carries record-index / byte-offset state.

Added

• Structured positional context on every parse error — record index, absolute and record-relative byte offsets, 001 control number, field tag (and indicator position or subfield code where applicable), offending bytes (capped at 32), and source filename. New MarcError::detailed() multi-line diagnostic includes a hex-dump of the byte window around the error offset; the default Display is a one-liner. Both produce byte-for-byte identical output across Rust and Python via matching _format() / detailed() methods on every Python exception class.
• Typed Python exception subclasses for every MarcError variant (e.g. InvalidIndicator, BadSubfieldCode, TruncatedRecord, XmlError, JsonError, WriterError, FatalReaderError) extending the closest pymarc-named parent so existing pymarc-style except clauses keep catching the same conditions. Each carries positional kwargs, supports pickle round-trip, and survives the PyO3 boundary with all attributes intact.
• Stable error codes (E001–E007, E099, E101, E105, E106, E201, E202, E301, E401, E402, E404) on every variant via MarcError::code() / slug(), with matching code / slug / help_url() on every Python class. Codes never get renumbered (policy in CONTRIBUTING.md); MRRC_DOCS_BASE_URL overrides the help-URL host. All codes documented in docs/reference/error-codes.md.
• Structured error serialization via to_dict() / to_json() on every Python exception (suitable for ELK / Datadog / Splunk) and MarcError::to_json_value() / to_json() on the Rust side; bytes fields hex-encoded, _cause flattens the exception chain, schema_version: 1 for forward-compat. New BytesNear public struct exposes the captured byte window.
• with_source() / from_path() builder methods on MarcReader, AuthorityMarcReader, and HoldingsMarcReader. from_path populates the source name from the file path so emitted errors carry the filename.
• Per-stream recovered-error cap (#110) via with_max_errors(n) on all three readers. In Lenient / Permissive modes the reader counts each recovered failure and halts with MarcError::FatalReaderError (E099) once the cap (default 10000) trips. Pass 0 to disable.
• Per-field lenient/permissive recovery parity across all three ISO 2709 readers (#121, #122). AuthorityMarcReader and HoldingsMarcReader now honor the same per-field error sites the bibliographic reader does (bad field-length / start-position digits, field-extends-past-data, data-field-too-short-for-indicators, parse_data_field failure). Strict-mode behavior unchanged at every site; lenient/permissive mode skips the offending entry, counts the recovery against the per-stream cap, and continues. Truncated-record dispatch in lenient/permissive mode no longer returns a hard error on a short read; instead it notes the recovery and falls through to best-effort directory parsing.
• 8 property tests (tests/properties.rs) covering binary, MARCXML, and MARCJSON round-trips plus four ISO 2709 structural invariants (leader length, directory tiling, indicator byte set, subfield code shape). ProptestConfig { cases: 64 }; full suite ~3s locally. Primer at docs/contributing/formal-methods.md (#111).
• Coverage-guided fuzzing (#90, #115). Standalone fuzz/ Cargo workspace with cargo-fuzz; nightly CI matrix runs parse_record (full reader) and roundtrip_binary (reader → writer → reader coupling) for 5 minutes daily at 03:00 UTC. Findings are not a PR gate; reproducers get copied into tests/data/fuzz-regressions/ to run on every PR. Triage playbook in docs/contributing/fuzzing.md.
• docs extra in pyproject.toml for mkdocs site dependencies; CI builds via uv sync --extra docs --no-install-project so the docs-only job doesn't need a Rust toolchain.

Changed

• Shared ISO 2709 parsing primitives + generic skeleton (#125). src/iso2709.rs owns the leader read, truncation-aware record-data read, single-entry directory parsing, ASCII numeric helpers, and the control-field-tag predicate. New iso2709_skeleton::Iso2709Builder trait + parse_iso2709_record<R, B> skeleton drives one record's parse end-to-end; each reader implements Iso2709Builder via a small adapter (~60 lines each) so read_record collapses to a one-line dispatch. ~200 lines of near-duplicate read_record body across three readers replaced by one shared implementation. New recovery::RecoveryCap struct consolidates the per-stream cap state machine that had been duplicated. Behavior changes that fall out of the unification: AuthorityMarcReader now treats too-short data fields as a strict-Err / lenient-skip event with cap accounting (was silently skipped in all modes); HoldingsMarcReader's field-extends-beyond-data lenient branch now salvages a clamped slice (matching bib); the holdings "field exceeds data" error message changed wording to match the other two readers. The skeleton is <R: Read, B: Iso2709Builder> with no dyn dispatch so trait calls monomorphize and inline at every call site, preserving hot-loop characteristics. Per-type quirks (authority's tag UTF-8 strictness + trailing 0x1F trim, holdings' strict UTF-8 on control fields) preserved via trait-method overrides; the wider strict-vs-lossy unification is tracked separately as a follow-up.
• HoldingsMarcReader::with_max_errors is now active. Originally landed inert when the cap was introduced (no recovery sites in the holdings path); the recovery sites added in this cycle hook into it.
• MarcError source-error chain walks correctly for IoError, XmlError, JsonError (was previously empty). Code that was checking Error::source() == None may need updating.
• Pinned Rust toolchain to 1.95.0 (#96, #97) via rust-toolchain.toml. Library MSRV (Cargo.toml → rust-version = "1.71") is unchanged.
• CI: skip workflows on docs-only changes. Added paths-ignore for **.md, docs/**, mkdocs.yml, LICENSE, and .gitignore to lint, test, build, python-build, benchmark-python, and benchmark-rust workflows. A docs-only push or PR previously fired ~47 jobs across the six workflows; it now fires zero. Mixed PRs (code + docs) still run normally — paths-ignore skips only when every changed path matches.
• .cargo/check.sh builds the docs site (mkdocs build) in full mode, surfacing broken cross-links pre-push. Skipped under --quick. Requires the docs extra (uv sync --all-extras).
• mkdocs warnings cleanup. Excluded docs/history/ (archival per CLAUDE.md) from the published site, removed its nav entry, and fixed broken cross-links and a stale anchor in active docs. Build warnings down from 18 to 4.

Fixed

• Python typing fidelity improvements. Stub gaps in mrrc/_mrrc.pyi closed (parse_batch_parallel / parse_batch_parallel_limited signatures, Field.delete_subfield, Record.to_marc21, module-level __version__) and several wrapper-side narrowing issues fixed. mypy mrrc/ and pyright mrrc/ now both report zero errors and run in .cargo/check.sh full mode. Type-only — no runtime behavior change.
• MARCXML reader: missing XML 1.1 §2.11 end-of-line normalization in text and CDATA content (#112). Switched both arms of read_leaf_text from decode() to xml_content() so CR / CRLF / NEL / LSEP normalize to LF per spec. Domain impact is small (MARC field content rarely carries line separators) but the divergence from spec was real.
• Read-path performance regression from structured-error refactor (#117). The ParseContext refactor stopped the compiler from inlining parse_data_field, costing ~15-17% on read hot paths vs v0.7.6 (investigated in #116). Restored to within +9-11% of baseline via #[inline(always)] on parse_data_field paired with shrinking ParseContext::current_field_tag to Option<[u8; 3]>. The compact context is what lets forced inlining avoid ballooning L1-i cache usage on parallel workloads.
• CI: Clippy collapsible_match errors in src/bibframe/converter.rs after Rust stable advanced to 1.95.0 (#96, #97).
• CI: ASAN job failed with -Zbuild-std on stable; sets RUSTUP_TOOLCHAIN=nightly on the ASAN step (#105).
• CI: Miri job could not run insta snapshot tests; setting INSTA_WORKSPACE_ROOT skips the cargo metadata spawn (#106).
• CI: docs deploy broke when #130 switched the install line to build mrrc itself; now uses uv sync --extra docs --no-install-project (#131).

Dependencies

• Bump ruff from 0.15.11 to 0.15.12
• Bump mypy from 1.20.1 to 1.20.2
• Bump pyright from 1.1.408 to 1.1.409

[0.7.6] - 2026-04-14

Added

• MARCReader error-handling flags (#78, #80): pymarc-compatible kwargs for MARCReader:
  • to_unicode: accepted for compatibility, warns if False (mrrc always converts to Unicode).
  • permissive: yields None for bad records instead of raising, matching pymarc behavior.
  • recovery_mode: exposes mrrc's RecoveryMode for salvaging partial data from malformed records.
  • Thanks to @acdha for filing #78.

Fixed

• Repeated control fields silently dropped (#77, #79): Control fields (e.g., multiple 007s) were stored in IndexMap<String, String>, causing later values to overwrite earlier ones. Changed to IndexMap<String, Vec<String>> across all three record types (Record, AuthorityRecord, HoldingsRecord). Updated all 7 serialization formats (ISO 2709, JSON, MARCJSON, MARCXML, CSV, Dublin Core/MODS, BIBFRAME) to emit all values. The pymarc-compatible get_fields() API now correctly returns all repeated control fields, matching pymarc behavior. Thanks to @acdha for reporting.
• delete_subfield() did not actually delete (#81, #82): Was returning the value without removing the subfield. Added Rust Field::delete_subfield() method, exposed via PyO3, and updated the Python wrapper to delegate. Now matches pymarc behavior.
• recovery_mode validation for authority/holdings readers (#82): Authority and holdings readers now reject invalid recovery_mode values with ValueError instead of silently defaulting to strict.

Changed

• Simplified control field API: Removed get_control_field_values() accessor from Record — redundant with direct access to the public control_fields map. get_control_field() remains as a convenience for non-repeatable tags (001, 003, 005, 008).
• Reduced code duplication across Python bindings and Rust core (#82):
  • Extracted _wrap_field() and _wrap_record() helpers in the Python wrapper, replacing 16+ inconsistent wrapping patterns.
  • Extracted shared reader_helpers.rs for PyO3 reader backends, reducing ~225 lines of copy-paste between authority and holdings readers.
  • Extracted control_field_char_at() utility for 008 field parsing, replacing 5 duplicated patterns across authority and holdings record types.

Dependencies

• Bump ruff from 0.15.8 to 0.15.10
• Bump mypy from 1.19.1 to 1.20.1
• Bump pytest from 9.0.2 to 9.0.3
• Bump softprops/action-gh-release from 2 to 3
• Bump actions/upload-pages-artifact from 4 to 5

[0.7.5] - 2026-04-05

Added

• Record.get(tag) for pymarc compatibility: Dict-like record.get('245') returns the first matching field or a default value, mirroring pymarc's Record.get(). Delegates to existing get_field().
• Field.is_control_field() and ControlField.is_control_field() for pymarc compatibility: Returns False on data fields and True on control fields, matching pymarc's unified Field.is_control_field() API.
• Record.__str__ and Record.__repr__: The Python Record wrapper now delegates to the Rust implementation instead of showing the default <mrrc.Record object at 0x...>. str(rec) returns Record(type=a) and repr(rec) returns <Record type=a fields=N>.
• SubfieldPatternQuery negation flag (#63, #73): SubfieldPatternQuery("020", "a", r"^978-", negate=True) pushes pattern inversion into Rust so non-matching fields never cross the FFI boundary. Thanks to @acdha@code4lib.social for suggesting this.
• SubfieldValueQuery negation flag (#64, #74): SubfieldValueQuery("650", "a", "History", negate=True) pushes value inversion into Rust. Works with both exact and partial matching.
• Comprehensive pymarc API compatibility (#71, #72): Full drop-in replacement for pymarc. Thanks to @mistersql@mastodon.social for highlighting several of these compatibility gaps. Changes include:
  • Record accessors as @property: All 17 record accessors (title, author, isbn, issn, subjects, location, notes, publisher, uniform_title, sudoc, issn_title, issnl, pubyear, series, physical_description, plus aliases physicaldescription, uniformtitle, addedentries) are now properties, matching pymarc's record.title syntax.
  • Unified ControlField into Field: Field('001', data='12345') creates a control field. ControlField remains as a backward-compatible subclass. Control field content accessed via .data attribute.
  • Record['xxx'] raises KeyError for missing tags (use record.get(tag) for safe access), matching pymarc behavior.
  • Record.as_marc() / as_marc21(): Returns ISO 2709 bytes.
  • Record.as_json() / as_dict(): pymarc-compatible MARC-in-JSON serialization.
  • Field.value(): Space-joined subfield values.
  • Field.format_field(): Human-readable text representation.
  • Field.as_marc() / as_marc21(): Field-level binary serialization.
  • Field.add_subfield(code, value, pos=N): Positional insert support.
  • Field.linkage_occurrence_num(): Extract $6 linkage info.
  • Field.convert_legacy_subfields(): Classmethod for old flat-list format.
  • add_field(*fields): Accepts multiple fields at once.
  • remove_field(*fields): Accepts multiple fields, returns None.
  • remove_fields(*tags): Bulk removal by tag.
  • add_ordered_field(*fields): Tag-sorted insert.
  • add_grouped_field(*fields): Insert after same-tag group.
  • parse_xml_to_array(), parse_json_to_array(), map_records(): Module-level convenience functions.
  • MARC constants: LEADER_LEN, DIRECTORY_ENTRY_LEN, END_OF_FIELD, END_OF_RECORD, SUBFIELD_INDICATOR, MARC_XML_NS, MARC_XML_SCHEMA.
  • Exception hierarchy: MrrcException base class with RecordLengthInvalid, RecordLeaderInvalid, BaseAddressInvalid, BaseAddressNotFound, RecordDirectoryInvalid, EndOfRecordNotFound, FieldNotFound, FatalReaderError, and BadSubfieldCodeWarning.
  • pubyear returns str (not int), matching pymarc.

Breaking Changes

• Record accessors are now @property: record.title() → record.title. Affects all 17 accessors: title, author, isbn, issn, issn_title, issnl, subjects, notes, location, series, sudoc, publisher, pubyear, physical_description, uniform_title, plus new physicaldescription, uniformtitle, addedentries.
• pubyear returns str: Previously returned Optional[int], now returns Optional[str] to match pymarc.
• Record['xxx'] raises KeyError for missing tags: Previously returned None. Use record.get(tag) for safe access that returns None.
• ControlField unified into Field: ControlField is now a subclass of Field. Control field content is in .data (not .value). Field('001', data='12345') is the preferred constructor.
• remove_field() returns None: Previously returned a list of removed fields.
• add_field() / remove_field() accept *args: Signature changed from single field to *fields. Existing single-arg calls still work.
• Default Field indicators changed from '0' to ' ': Matches pymarc. Only affects fields created without explicit indicators.

Changed

• SubfieldPatternQuery and SubfieldValueQuery marked #[non_exhaustive] (#74): Prevents external struct literal construction, requiring use of constructors. Future field additions will not be semver-breaking.

Dependencies

• Bump pygments from 2.19.2 to 2.20.0 (#67)
• Bump ruff from 0.15.4 to 0.15.8
• Bump actions/deploy-pages from 4 to 5
• Bump codecov/codecov-action from 5 to 6

Fixed

• mrrc.__version__ now reports the correct version: Previously hardcoded as "0.1.0" in both mrrc/__init__.py and src-python/src/lib.rs. Now derived from Cargo.toml at compile time via env!("CARGO_PKG_VERSION"), eliminating the need to manually update version strings during releases.

Documentation

• Migration guide updated: Added record.get() and field.is_control_field() examples to docs/guides/migration-from-pymarc.md, reflecting closer pymarc API parity.
• All docs and examples updated for pymarc API compatibility: Record accessors now shown as properties (record.title not record.title()), control field access uses .data, missing-field access patterns use record.get() or try/except, and new methods/constants/exceptions are documented throughout the API reference, migration guide, quickstart, tutorials, and runnable examples.

[0.7.4] - 2026-03-04

Added

• Proptest binary round-trip property test (#39, #42): New tests/properties.rs uses proptest to generate arbitrary structurally valid MARC records and verify that serialization to ISO 2709 and parsing back produces identical results. Covers leader fields, control fields, and data fields with indicators and subfields.
• Cargo-semver-checks CI workflow (#37, #40): New .github/workflows/semver.yml runs cargo-semver-checks on PRs that touch src/ or Cargo.toml to catch accidental semver-breaking changes.
• Nightly Miri CI workflow (#38, #41): New .github/workflows/miri.yml runs cargo +nightly miri test --lib on a daily schedule to detect undefined behavior.
• serialization_never_panics property test (#44, #46): New proptest verifying MarcWriter never panics or errors on any generated record, catching serialization bugs the round-trip test might miss.
• Deduplicate control field tags in proptest (#43, #45): arb_record() now uses a HashSet to skip duplicate control field tags, ensuring generated records are structurally valid per MARC.

Changed

• check.sh --quick now skips Python tests (#53, #56): --quick previously skipped the maturin build but still ran Python tests against a stale extension. Now --quick runs rustfmt, clippy, ruff, Rust tests, and doc tests (~10s). Full mode adds doc check, audit, maturin build, and Python tests (~16s).

Fixed

• Serialization functions now accept wrapped Python Records (#57, #58): record_to_json(), record_to_xml(), record_to_marcjson(), record_to_mods(), record_to_dublin_core(), and record_to_dublin_core_xml() previously rejected Record objects returned by xml_to_record() and other deserialization functions with TypeError: 'Record' object is not an instance of 'Record'. The PyO3 functions expected the raw _mrrc.Record type but received the Python wrapper mrrc.Record. All six functions now use a shared extract_record() helper that accepts both types, matching the pattern already used by record_to_csv().
• record_to_dublin_core_xml now exported from Python wrapper (#59): The function existed in the Rust extension (_mrrc) but was missing from mrrc/__init__.py imports and __all__, making it inaccessible as mrrc.record_to_dublin_core_xml().
• Skip rayon tests under Miri (#47, #49): Annotate 4 rayon_parser_pool tests with #[cfg_attr(miri, ignore)] to work around a known stacked borrows violation in crossbeam-epoch 0.9.18 (crossbeam-rs/crossbeam#1181). Tracking re-enablement in #48.
• Arithmetic overflow panic on malformed leaders (#32): MarcReader, AuthorityReader, and HoldingsReader now return Err(MarcError::InvalidLeader) instead of panicking when record_length or data_base_address in the leader is less than 24. New Leader::validate_for_reading() method performs the check in all three binary readers.
• Python examples used Python file I/O instead of Rust I/O (#53, #54): All Python examples now pass file path strings to MARCReader/MARCWriter instead of open() file objects, using the Rust I/O backend which releases the GIL. Updated type stub docstring examples to match.

Documentation

• Agent docs overhaul (#50): New CLAUDE.md with project overview, key files, build/test commands, and architecture reference. Rewritten AGENTS.md replaces inception-era framing, migrates all bd references to br.
• Docs navigability improvements (#51): Added _mrrc.pyi type stub admonition to Python API reference page. Moved Design and History under Contributing in mkdocs nav.
• Context7 configuration (#52): New context7.json configuring documentation indexing for source, bindings, Python package, docs, and examples.
• Migrate bd references to br (#55): Migrated .github/copilot-instructions.md and docs/contributing/release-procedure.md from bd to br commands, with explicit br sync --flush-only + git commit workflow.
• Updated migration guide (docs/guides/migration-from-pymarc.md) to recommend path-based MARCReader input instead of Python file objects, with comments explaining GIL release and multi-thread parallelism benefits.

[0.7.3] - 2026-02-26

Added

• Standard MARCXML support (#15): xml_to_record() and record_to_xml() / to_xml() now produce and parse standard LOC MARCXML. Output includes XML declaration, xmlns="http://www.loc.gov/MARC21/slim" namespace, and tag/ind1/ind2/code as XML attributes (not child elements). Parsing accepts all three namespace forms: default namespace (<record xmlns="...">), prefix namespace (<marc:record xmlns:marc="...">), and no namespace (<record>). New xml_to_records() function parses <collection> wrappers containing multiple records. Rust module renamed from xml to marcxml (record_to_marcxml(), marcxml_to_record(), marcxml_to_records()); Python API names unchanged for pymarc compatibility.
• get_linked_fields() for 880 alternate script field linkage (#19): New pymarc-compatible record.get_linked_fields(field) returns all 880 fields linked via subfield $6 occurrence matching. Also adds get_linked_field() (singular), get_original_field() (reverse lookup from 880 to original), and get_field_pairs(tag) (original + 880 tuples). Fixed LinkageInfo parser to handle real MARC script identification codes: (2 (Hebrew), (3 (Arabic), $1 (CJK), (N (Cyrillic), (S (Greek). Includes tests for Hebrew, Arabic, Chinese, Russian, and Greek scripts plus linkage across multiple field types (title, author, publisher, subject, series, notes, added entries).
• MARCReader.backend_type property: Exposes the reader's I/O backend as a string ("rust_file", "cursor", or "python_file") for diagnostics and testing.
• Dependabot configuration: Enabled automated dependency updates for cargo, uv, and github-actions ecosystems (weekly schedule).
• GIL release verification tests: Deterministic tests proving py.detach() actually releases the GIL during record parsing. Uses sys.setswitchinterval(100.0) to suppress automatic GIL switching, then verifies a background counter thread makes progress — which can only happen if the GIL is explicitly released. Covers all three backends: rust_file, cursor, and python_file.

Changed

• Removed 22 duplicate inherent methods from Record (#26): Helper methods like title(), publisher(), isbn(), etc. were defined both as inherent methods on Record and on the RecordHelpers trait. The inherent methods shadowed the trait, causing maintenance burden and confusion. Removed all duplicates; callers now use the RecordHelpers trait (already re-exported in the public API). Rust users may need to add use mrrc::RecordHelpers; if not already imported.
• Replaced flaky GIL release test with deterministic verification (#3): The test_pathlib_path_threading_equivalent_to_str test used timing-based speedup ratios to verify GIL release, which was unreliable on shared CI runners. Replaced with TestBackendTypeRouting (6 tests asserting each input type routes to the correct backend) and TestPathlibThreadingCorrectness (2 tests verifying parallel reads produce correct results without speedup assertions).
• quick-xml 0.31 → 0.39 (#7): Migrated MODS XML parser to quick-xml 0.39 API. BytesText::unescape() replaced by decode() with explicit entity resolution via new Event::GeneralRef handler (predefined XML entities and numeric character references). Reader::trim_text() moved to Reader::config_mut().trim_text().
• pyo3 0.27 → 0.28 (#8): Migrated PyO3 bindings to 0.28 API for free-threaded Python support. Python::with_gil() → Python::attach(), Python::assume_gil_acquired() → Python::assume_attached(). Added from_py_object annotation to all 10 #[pyclass] types implementing Clone.
• thiserror 1.0 → 2.0 (#6): Updated error derive macro.
• nom 7.1 → 8.0 (#5): Updated parser combinator library.
• ruff 0.14.14 → 0.15.2 (#9): Updated Python linter.
• GitHub Actions updates: actions/checkout 4 → 6 (#14), codecov/codecov-action 4 → 5 (#13), astral-sh/setup-uv 4 → 7 (#12), actions/download-artifact 4 → 7 (#11), moonrepo/setup-rust 0 → 1 (#10), actions/setup-python 5 → 6 (#24), actions/upload-artifact 4 → 6 (#23), actions/upload-pages-artifact 3 → 4 (#22), actions/cache 4 → 5 (#21), softprops/action-gh-release 1 → 2 (#20).

Fixed

• Format conversion functions now return wrapped Python objects (#16): marcjson_to_record(), json_to_record(), xml_to_record(), mods_to_record(), and mods_collection_to_records() previously returned raw Rust PyRecord objects. Field subscripting (field['a']), get_subfields(), and leader indexing (leader[9]) all failed on these records. Now all format conversion functions wrap results in the Python Record class, matching the behavior of records from MARCReader.
• publisher() now checks 264 (RDA) as fallback (#17): Previously only returned $b from field 260. Now falls back to field 264 with indicator2='1' (publication) for RDA-cataloged records. Also updated place_of_publication(), publication_date(), publication_info(), and publication_year() / pubyear() to check 264 as fallback, matching pymarc behavior.
• subjects() now covers all 6XX fields (#18): Previously only returned $a values from field 650 (topical term). Now matches pymarc's subjects() coverage: 600, 610, 611, 630, 648, 650, 651, 653, 654, 655, 656, 657, 658, 662, 690-691, 696-699. Also updated subjects_with_subdivision() and subjects_with_note() to search all subject fields.
• Removed flaky timing assertion from test_backend_comparison_1k: The benchmark's assert median_rustfile < median_pythonfile * 2 failed intermittently on shared CI runners due to disk I/O variance. Timing data is still printed for informational purposes.

Documentation

• Updated Python quick example in docs/index.md to use path string instead of file I/O wrapper, demonstrating the GIL-releasing code path.

[0.7.2] - 2026-02-11

Added

• MODS XML read support: mods_to_record(xml_str) and mods_collection_to_records(xml_str) parse MODS XML (single record or <modsCollection>) into MARC Record objects. Covers titleInfo, name, subject, originInfo, physicalDescription, abstract, note, genre, classification, location, relatedItem, recordInfo, identifier, accessCondition, tableOfContents, targetAudience, and language elements with conformance tests using LOC-derived fixtures.
• Cross-compiled Linux wheels (mrrc-experiments#16): Release workflow now builds aarch64 and i686 Linux wheels via maturin-action Docker cross-compilation, increasing wheel count from 15 to 25 per release.
• Field constructor subfields= and indicators= kwargs (mrrc-experiments#15): Field now accepts subfields= (list of Subfield) and indicators= (list/tuple of two strings) as keyword arguments, enabling inline construction matching pymarc style: Field('245', indicators=['0', '1'], subfields=[Subfield('a', 'Title')]). The Rust binding already supported this signature; the Python wrapper now passes through both kwargs.
• Record constructor fields= kwarg (mrrc-experiments#15): Record now accepts an optional fields= keyword argument (list of Field), enabling inline record construction: Record(fields=[Field(...), ...]). This goes beyond pymarc parity (pymarc's Record.__init__ does not accept fields=) as a UX improvement. Record() with no arguments also now works, defaulting to Leader().
• Constructor kwargs tests: 8 new tests in TestConstructorKwargs covering indicators=, subfields=, fields=, combined usage, full inline construction, and backward compatibility.
• PEP 561 py.typed marker: Added to root mrrc/ package so type checkers recognize shipped type stubs.
• Project layout documentation: New docs/contributing/project-layout.md describes the three-layer architecture (core Rust → PyO3 bindings → Python package), how maturin and the Cargo workspace connect them, directory structure, build commands, and common development workflows.
• Pre-push git hook: .githooks/pre-push runs .cargo/check.sh before every push, preventing code from reaching CI that would fail local checks. Opt-in via git config core.hooksPath .githooks.
• check.sh --quick flag: Skips docs, audit, and maturin build for faster inner-loop iteration (keeps fmt, clippy, tests, ruff).

Changed

• GitHub releases now include changelog notes: python-release.yml auto-extracts the relevant CHANGELOG.md section and includes it in the GitHub Release body. Backfilled v0.7.1 release notes.
• Pymarc compatibility tests fully enabled: Removed 10 pytest.skip guards from test_pymarc_compatibility.py — all features (to_json, to_xml, to_marcjson, to_dublin_core, to_marc21, writer, roundtrip, test data) were already implemented. 88/88 tests now pass with 0 skipped.
• Documentation updated for inline construction: Quickstart, API reference, migration guide, writing tutorial, and examples now show Field(subfields=..., indicators=...) and Record(fields=...) as the primary construction pattern, with add_subfield()/add_field() as the incremental alternative. Migration guide updated to reflect Record() no longer requires an explicit Leader argument and field creation is now closer to pymarc.
• Type stubs updated: Field.__new__ and Record.__new__ in .pyi now include the new keyword-only parameters.
• Type stubs enriched: Merged rich docstrings into mrrc/_mrrc.pyi from the old src-python stubs — Leader properties with MARC position descriptions, MARCReader GIL/concurrency/thread-safety docs, MARCWriter context manager protocol, BibframeConfig setter methods and property getters, RdfGraph parse()/triples()/__len__(). Added mods_to_record and mods_collection_to_records stubs.
• Removed stale src-python/python/ directory: Deleted redundant Python package directory (1,235 lines) left over from the Phase 5 layout migration. Root mrrc/ is now the sole Python package location.
• Test suite audit: Deleted 5 redundant/inert files from src-python/tests/, migrated 4 test files to tests/python/ (backend parity, type detection, rayon parser pool, record boundary scanner), extracted 1 pipeline regression test. Deleted src-python/tests/ entirely. Python test count: 364 → 433.
• Ruff lint compliance: Fixed 76 ruff errors across mrrc/ and tests/python/ (bare except, unused imports/variables, walrus operator patterns, f-string placeholders, true/false comparisons). Added ruff check step to check.sh.
• Rust integration tests in check.sh: Changed cargo test --lib to cargo test --lib --tests, adding 17 integration test files (bibframe, mods, field query, concurrent GIL, etc.) to the local CI script.
• CI workflow alignment with check.sh: Closed gaps where CI was missing checks that check.sh runs locally. Added clippy on mrrc-python to lint.yml (was only linting mrrc core). Added ruff Python lint job to lint.yml (via uvx). Fixed rustdoc scope from --package mrrc to --all to include PyO3 bindings. Changed test.yml from --lib --bins to --lib --tests --package mrrc, adding 15+ integration test files that were skipped in CI. Removed unused mypy/pyright install from python-build.yml test-wheels job. Upgraded actions/cache from v3 to v4 across all workflows.
• CI efficiency improvements: Switched all pip install calls to uv pip install --system across python-build.yml, benchmark-python.yml, and python-release.yml for faster dependency resolution. Consolidated three separate cargo cache actions (registry, index, build) into single multi-path caches across five workflows. Switched cargo-audit and cargo-tarpaulin from cargo install (compiles from source) to pre-built binaries via taiki-e/install-action. Combined two tarpaulin runs into one (--out Xml Html). Upgraded codecov/codecov-action from v3 to v4, actions/setup-python from v4 to v5. Removed redundant pip install maturin and single-element Rust version matrix from build.yml.

Fixed

• Rust formatting: Applied cargo fmt to src/mods.rs and src-python/src/formats.rs (pre-existing match arm brace style).
• Clippy doc lint: Escaped OCoLC in tests/mods_conformance_tests.rs doc comment to satisfy clippy::doc_markdown.
• Flaky CI benchmark: test_bytesio_vs_file_isolation switched from mean to median for I/O overhead measurement, preventing single-iteration CI runner spikes from failing the test. Removed hard assertion (test value is diagnostic, not a correctness gate).

[0.7.1] - 2026-02-10

Changed

• BREAKING: Removed 7 experimental serialization formats (mrrc-experiments#19): Deleted protobuf, arrow, parquet, flatbuffers, messagepack, cbor, and avro support (~15,100 lines across 78 files). These added significant build complexity and dependency weight without proven adoption. Only ISO 2709 and BIBFRAME remain.
• BIBFRAME promoted to core: BIBFRAME conversion is now always-compiled rather than feature-gated (format-bibframe), reflecting its importance as the LOC linked data format. Dependencies oxrdfio and oxrdf are now non-optional.
• Python bindings simplified: Removed format-specific classes (ProtobufReader, ArrowWriter, etc.), mrrc/analytics.py, and format submodules. Simplified read()/write() helpers and __init__.py exports.
• Clean repo history: Recreated repository as dchud/mrrc with a clean two-commit history, replacing dchud/mrrc-experiments. Build artifacts, large test fixtures, and accumulated cruft from the experimental phase are no longer in git history.
• Test fixtures: Removed 25MB 100k_records.mrc from repo; gitignored to prevent re-commit. Regenerable via scripts/generate_benchmark_fixtures.py. Benchmark tests depending on 100k fixture removed.
• Build simplification: Removed build.rs (was only for protobuf/flatbuffers code generation), src/generated/, proto/, and ~20 dependencies from Cargo.toml.
• Local CI script: Updated .cargo/check.sh to use uv run for maturin and pytest steps instead of manual venv activation.
• 100k fixture scrub: Removed remaining references to 100k_records.mrc from scripts, benches, CI workflow names, and public docs (benchmarks, contributing, tests README). Fixture generation script no longer produces the 100k file.
• Developer docs standardized on uv: Replaced manual venv activation and bare maturin develop/pytest commands with uv sync/uv run equivalents across installation, development setup, testing, and migration guides.
• Release procedure updated: Refreshed for OIDC trusted publishing (replaces PYPI_API_TOKEN), added installation.md and quickstart-rust.md to version-bump checklist.
• Rust dependency version in docs: Updated hardcoded mrrc = "0.6" to "0.7" in installation and quickstart guides.

Fixed

• Flaky benchmark test: test_backend_comparison_1k now uses 5 iterations with median instead of 3 iterations with mean, making it resilient to single-iteration CI runner I/O spikes.
• CI: Python release workflow: Added actions/setup-python before maturin-action to properly set Python version; python-version is not a valid input for maturin-action
• CI: Python release OIDC publishing: Removed password: secret from pypa/gh-action-pypi-publish step; action auto-detects OIDC token from id-token: write permission
• CI: Re-enabled ASAN memory-safety workflow: Upstream Rust issue (rust-lang/rust#144168) that caused zerocopy nightly incompatibility has been resolved
• CI: ASAN workflow zerocopy_derive fix: Removed -Zavoid-dev-deps flag which caused zerocopy_derive proc-macro resolution failures on nightly
• CI: CodSpeed benchmark integration: Re-linked repository with CodSpeed after clean repo creation
• Beads: Gitignore rotated daemon logs: Added daemon-*.log.gz pattern to .beads/.gitignore and removed accidentally committed 2.7MB rotated log file

[0.7.0] - 2026-02-05

Added

Testing (2026-02-04)

• pathlib.Path GIL Release Tests: Added tests verifying that pathlib.Path objects achieve the same GIL release behavior as string paths
  • test_pathlib_path_sequential_baseline: Baseline sequential reading with Path objects
  • test_pathlib_path_threading_equivalent_to_str: Verifies Path and str have equivalent threading speedup (within 30%)
  • test_pathlib_vs_str_equivalent_performance: Confirms both use the same RustFile backend
  • Location: tests/python/test_pathlib_gil_release.py

Changed

Python Version Support (2026-02-03)

• Dropped Python 3.9 support: Python 3.9 reached end-of-life in October 2025
• Added Python 3.13 support: Stable release from October 2024
• Added Python 3.14 support: Stable release from October 2025
• Updated minimum Python version from 3.9 to 3.10 in pyproject.toml, CI workflows, and documentation
• Note: RHEL 9 users should use dnf module enable python3.11 or containers for Python 3.10+

Fixed

Security (2026-02-03)

• bytes crate upgrade: Upgraded bytes from 1.9.0 to 1.11.1 to address RUSTSEC-2026-0007

CI/CD Improvements (2026-02-03)

• CodSpeed Rust Benchmark Integration (PR #8): Added continuous performance tracking for Rust benchmarks
  • Integrated codspeed-criterion-compat v4.3.0 as drop-in replacement for criterion
  • New benchmark-rust.yml workflow runs Rust benchmarks via cargo codspeed
  • Added CodSpeed badge to README.md
• Benchmark Workflow Naming Consistency: Renamed workflows for clarity
  • python-benchmark.yml → benchmark-python.yml ("Benchmarks: Python")
  • codspeed.yml → benchmark-rust.yml ("Benchmarks: Rust")
  • Groups benchmark workflows alphabetically, makes target language clear at a glance

CI/CD Fixes (2026-02-02)

• CodSpeed Action v4 Migration: Fixed benchmark workflow after upgrading from deprecated v1
  • Added required mode: walltime parameter for standard CI runners
  • CodSpeed v4 now requires explicit mode selection ('simulation' or 'walltime')

Documentation

Documentation Updates (2026-02-04)

• Fixed benchmark documentation links: Updated internal links in index.md, query-dsl.md, and architecture.md to use lowercase filenames
• Testbed Design Document (docs/design/ideas-for-test-projects.md):
  • Added comprehensive state management design (YAML + SQLite hybrid)
  • Added interaction models section (centralized vs local/private testing)
  • Added repository growth timeline and pruning strategies
  • Added MkDocs documentation structure with Divio-style organization
  • Updated project phases (7 phases including documentation and initial data)
  • Consolidated duplicate sections for cleaner document structure

Documentation Improvements (2026-02-02)

• File Renames and Reorganization:
  • Renamed streaming-large-files.md → working-with-large-files.md to better reflect content scope
  • Renamed benchmark docs to lowercase convention: RESULTS.md → results.md, FAQ.md → faq.md, README.md → index.md
  • Updated all cross-references and mkdocs.yml navigation
• Fixed Broken Links:
  • Fixed README.md tutorial links that were returning 404s
  • Updated to point to actual tutorial paths (tutorials/python/reading-records/, tutorials/rust/reading-records/)
• Formatting Fixes:
  • Fixed bullet list formatting in threading-python.md, performance-tuning.md, results.md, migration-from-pymarc.md
  • Added blank lines after colons before bullet lists (required by some markdown parsers)
• Content Improvements:
  • Added uv example to installation.md for building from source
  • Added hardware info (2025 MacBook Air M4) to performance-tuning.md
  • Added Rust/Python format name columns to bibframe RDF serialization table
  • Added "Same?" column to migration-from-pymarc.md API comparison tables for clarity
  • Removed emojis from migration-from-pymarc.md section headers
  • Fixed encoding.md: removed non-existent detect_encoding import, noted EncodingValidator is Rust-only
  • Simplified python-api.md format note (all formats bundled in wheel, no extra steps)
  • Added feature-gated format installation link to formats.md

Documentation Reorganization (2026-01-29)

• Complete restructure using Material for MkDocs:
  • New tabbed navigation with Getting Started, Tutorials, Guides, Reference, Examples sections
  • Light/dark theme toggle with persistent preference
  • Built-in search with instant results
  • Mobile-responsive layout
• Getting Started:
  • quickstart-python.md: 5-minute Python quickstart with installation, reading, writing examples
  • quickstart-rust.md: 5-minute Rust quickstart with Cargo setup and basic operations
• Tutorials (split from monolithic guides):
  • Python tutorials: reading-records, writing-records, querying-fields, format-conversion, concurrency (5 files)
  • Rust tutorials: reading-records, writing-records, querying-fields, format-conversion, concurrency (5 files)
• Reference Documentation:
  • marc-primer.md: MARC record structure explained for non-librarians (leader, directory, fields, indicators, subfields)
  • encoding.md: Character encoding reference (MARC-8, UTF-8, escape sequences, detection)
  • python-api.md: Complete Python API reference (Record, Field, MARCReader, formats, BIBFRAME)
  • rust-api.md: Complete Rust API reference (types, builders, traits, feature flags)
• Examples Index: Organized links to 29 example files by category (Python/Rust)
• README Reduction: Streamlined from 897 to 115 lines per documentation best practices
• Archived Legacy Docs: Moved superseded files to docs/history/legacy-docs-superseded/

[0.6.0] - 2026-01-29

Added

BIBFRAME Conversion (2026-01-28, updated 2026-01-29)

• Bidirectional MARC ↔ BIBFRAME conversion (format-bibframe feature):
  • marc_to_bibframe(): Convert MARC bibliographic records to BIBFRAME 2.0 RDF graphs
  • bibframe_to_marc(): Reverse conversion from BIBFRAME RDF back to MARC records
  • Full implementation of LOC MARC21 to BIBFRAME 2.0 Conversion Specifications
  • Support for all MARC content types: books, serials, music, maps, visual materials
• Python BIBFRAME Bindings (2026-01-29):
  • mrrc.marc_to_bibframe(record, config): Python wrapper for MARC→BIBFRAME conversion
  • mrrc.bibframe_to_marc(graph): Python wrapper for BIBFRAME→MARC conversion
  • mrrc.BibframeConfig: Configuration class with all options exposed
  • mrrc.RdfGraph: RDF graph class with serialize() method for all formats
  • mrrc.RdfFormat: Enum for output format selection (RdfXml, NTriples, Turtle, JsonLd)
• Hub/Expression Support (2026-01-29):
  • MARC 240 (Uniform Title) creates bf:Hub for expression-level grouping
  • Work → hasExpression → Hub → hasInstance → Instance linking pattern
  • Hub → expressionOf → Work reverse relationship
  • Supports translations, versions, and other expression-level variants
• Item Creation from Holdings (2026-01-29):
  • MARC 852 (Location) creates bf:Item entities linked to Instance
  • Call number from $h/$i, sublocation from $b, barcode from $p
  • MARC 876-878 (Item Information) support for detailed holdings
  • Multiple Items per Instance with sequential URI generation
• Classification Support (2026-01-29):
  • MARC 050 → bf:ClassificationLcc (Library of Congress Classification)
  • MARC 060 → bf:ClassificationNlm (National Library of Medicine)
  • MARC 080 → bf:ClassificationUdc (Universal Decimal Classification)
  • MARC 082 → bf:ClassificationDdc (Dewey Decimal Classification)
  • MARC 084 → bf:Classification (Other schemes with source in $2)
  • classificationPortion ($a) and itemPortion ($b) properties
• RDF Serialization Formats:
  • RDF/XML (W3C standard, maximum interoperability)
  • N-Triples (simple line-based triple format)
  • Turtle (human-readable, compact RDF syntax)
  • JSON-LD (modern web standard, JSON representation)
• Configuration Options (BibframeConfig):
  • Custom base URI for generated resources
  • Output format selection
  • Authority linking control
  • BFLC extension support
• Comprehensive Test Suite (111 tests):
  • 48 unit tests for individual MARC→BIBFRAME mappings (including Hub, Item, Classification)
  • 26 validation tests for BIBFRAME 2.0 ontology compliance
  • 16 integration tests with real-world record types
  • 16 round-trip tests documenting acceptable data loss
  • 5 baseline comparison tests against official LOC tool
• Examples:
  • Rust: marc_to_bibframe.rs, bibframe_to_marc.rs, bibframe_batch.rs
  • Python: marc_to_bibframe.py, bibframe_roundtrip.py, bibframe_config.py
• Documentation:
  • README section with conversion examples
  • API documentation with docstrings
  • Known limitations and data loss documentation

Changed

Field Insertion Order Preservation (2026-01-14)

• Record fields now preserve insertion order instead of sorting by tag
  • Uses IndexMap for field storage in Record, AuthorityRecord, and HoldingsRecord
  • Enables round-trip fidelity: serialization and deserialization now preserve original field order
  • Field iteration via fields() and control_fields_iter() returns fields in insertion order
  • Breaking change for code assuming tag-sorted order: Use fields_by_tag() for explicit tag-based access
• Unplanned changes during implementation:
  • fields_in_range() now iterates and filters (was using BTreeMap::range())
  • remove_fields_by_tag() now uses shift_remove() to preserve order of remaining fields
• Performance impact: ~17-22% regression in record parsing benchmarks (acceptable trade-off for round-trip fidelity)
  • Baseline: 9.5ms for 10k records → After: 11.3ms for 10k records
  • Still significantly faster than pymarc (~4x improvement maintained)

Performance Optimizations (2026-01-19)

• SmallVec for Subfield Storage:
  • Replaced Vec<Subfield> with SmallVec<[Subfield; 4]> for subfield arrays
  • Targets common case: 85-90% of real-world MARC records have ≤4 subfields per field
  • Performance gain: +4.6% roundtrip throughput (read+write cycle)
  • Zero-copy inline storage for typical records, automatic heap spillover for large records
  • Maintains API compatibility - transparent to users
• Parse Digits Optimization:
  • Eliminated string allocations in parse_digits() parser combinator
  • Reduced numeric field parsing overhead with direct byte-based validation
  • Contributes to overall +6.0% combined optimization gain (measured across parse + serialize pipeline)

Added

New Binary Format Support (2026-01-22)

• Apache Arrow Columnar Format (format-arrow feature):
  • ArrowWriter: Converts MARC records to Arrow RecordBatch for analytics workflows
  • ArrowReader: Reads Arrow IPC files back to MARC records with full fidelity
  • Preserves field sequence and control field data through round-trip conversion
  • Performance: 96% compression ratio, ~865k records/second throughput
  • Enables SQL queries over MARC data via DuckDB and Polars integration
• FlatBuffers Zero-Copy Format (format-flatbuffers feature):
  • FlatbuffersWriter: Serializes MARC records to FlatBuffers binary format
  • FlatbuffersReader: Zero-copy deserialization without parsing overhead
  • Ideal for memory-constrained environments and streaming applications
  • Performance: 64% memory savings vs JSON, 1M+ records/second zero-copy access
• MessagePack Compact Binary (format-messagepack feature):
  • MessagePackReader/MessagePackWriter for compact binary serialization
  • 25% smaller than JSON with equivalent structure preservation
  • Cross-platform interoperability with 50+ language implementations
  • Performance: ~750k records/second throughput
• FormatReader/FormatWriter Traits: Unified interface enabling consistent API across all formats

Python Format Bindings (2026-01-22)

• All Formats Available in Python: Arrow, FlatBuffers, MessagePack, and Protobuf readers/writers
• Format-Agnostic Helpers:
  • mrrc.read(path_or_data, format=None): Auto-detect format from file extension or content
  • mrrc.write(records, path, format=None): Write records to any supported format
• Type Stubs (.pyi files): Full IDE autocompletion and type checking support
• mrrc/formats/ Package: Organized modules for format-specific operations
  • mrrc.formats.marc: ISO 2709 reading/writing
  • mrrc.formats.protobuf: Protobuf serialization with schema evolution
  • mrrc.formats.arrow: Arrow/Parquet operations with analytics integration
  • mrrc.formats.flatbuffers: FlatBuffers zero-copy access
  • mrrc.formats.messagepack: MessagePack compact binary for APIs

Analytics Integration (2026-01-22)

• mrrc.analytics Module: SQL and DataFrame operations over MARC data
  • to_duckdb(records): Create DuckDB relation for SQL queries
  • to_polars(records): Create Polars DataFrame for data analysis
  • Enables filtering, aggregation, and joins across millions of records
• export_to_parquet() Helper: Convert Arrow tables to Parquet files for data lakes
• Integration Examples: Working examples demonstrating analytics workflows

Documentation Expansion (2026-01-22)

• New User Guides:
  • INSTALLATION_GUIDE.md: Complete installation instructions for Python and Rust, feature flags, platform-specific notes, troubleshooting
  • FORMAT_SELECTION_GUIDE.md: Decision tree for choosing the right format, comparison matrix, use case recommendations
  • PYTHON_TUTORIAL.md: Comprehensive Python tutorial covering reading, writing, field access, format conversion, Query DSL
  • RUST_TUTORIAL.md: Complete Rust tutorial with builder pattern, traits, parallel processing with Rayon
  • STREAMING_GUIDE.md: Large file handling patterns, O(1) memory streaming, parallel processing strategies
• Format Support Matrix: Added comprehensive format comparison table to README.md
• Enhanced Python Docstrings: All format modules include use cases, performance notes, and working examples

Format Evaluation & Strategy (2026-01-19)

• Binary Format Comparison: Evaluated MessagePack, CBOR, Avro, Arrow, FlatBuffers against library requirements
• Strategy Documentation: Decision rationale and benchmarks archived in docs/history/format-research/

Fixed

• Example Code Quality: Resolved clippy warnings in example code for clean builds
• Code Formatting: Applied rustfmt to all recent additions for consistency

Documentation Updates

• Benchmark Documentation Accuracy (2026-01-19):
  • Refreshed all benchmark measurements with latest performance data
  • Clarified that reported numbers are post-warm-up (JIT stabilized)
  • Updated comparison baselines to reflect recent optimizations
• Architecture Documentation: Enhanced documentation structure in docs/design/

[0.4.0] - 2026-01-09

Added

Query DSL Python Bindings (2026-01-11)

• Python Query DSL: Exposed Rust Query DSL to Python with full feature parity
  • FieldQuery: Builder pattern for complex field matching (tag, indicators, subfields)
  • TagRangeQuery: Match fields within a tag range (e.g., 600-699 for all subjects)
  • SubfieldPatternQuery: Regex matching on subfield values
  • SubfieldValueQuery: Exact or partial string matching on subfield values
• Record Query Methods: New methods on Record for advanced field searching
  • fields_by_indicator(tag, indicator1=None, indicator2=None): Filter by indicators
  • fields_in_range(start_tag, end_tag): Find fields within a tag range
  • fields_matching(query): Use FieldQuery objects for complex matching
  • fields_matching_range(query): Use TagRangeQuery for range-based matching
  • fields_matching_pattern(query): Regex matching via SubfieldPatternQuery
  • fields_matching_value(query): String matching via SubfieldValueQuery
• Query DSL Documentation: Comprehensive guide at docs/QUERY_DSL.md covering:
  • Philosophy: Why multiple query types (performance, clarity)
  • All query types with examples
  • Practical cataloging scenarios (LCSH filtering, ISBN-13, subject analysis)
  • Comparison table vs pymarc's get_fields()
• Query DSL Tests: 42 new tests in tests/python/test_query_dsl.py

Developer Experience (2026-01-11)

• Unified Testing Workflow: Single command (.cargo/check.sh) for full pre-push verification (~30s)
  • Runs rustfmt, clippy, documentation, security audit, maturin build, and Python tests
  • Uses pytest marker-based test selection (-m "not benchmark") to run 314 core tests in ~6s
  • Excludes 61 benchmark tests from default run (available via pytest -m benchmark)
  • Documented in AGENTS.md with command reference table and CI alignment
• CI Workflow Fixes:
  • Fixed ASAN memory-safety workflow: Exclude dev-dependencies (zerocopy) that have nightly Rust incompatibility
  • Fixed coverage workflow: Exclude PyO3 bindings package to avoid Python linker errors in tarpaulin

API Standardization (2026-01-12)

• Format Conversion API Naming: Standardized across all format modules for consistency
  • Added record_to_csv() for single-record CSV export (delegates to records_to_csv())
  • Documented records_to_csv() plural pattern: semantically correct for batch-oriented tabular format
  • All format modules now follow: record_to_format() (single) + records_to_format() (batch where applicable)
• CSV Export to Python: Exposed Rust CSV functions to Python bindings
  • record_to_csv(record): Export single record to CSV format
  • records_to_csv(records): Export multiple records to CSV (batch mode)
  • records_to_csv_filtered(records, filter_fn): Export with custom field filtering (filter_fn takes tag string, returns bool)
  • Handles both direct PyRecord and wrapped Record instances for flexibility
  • Completes API consistency with other format converters (JSON, XML, MARCJSON, Dublin Core, MODS)

Code Quality & Cleanup (2026-01-09)

• Memory Safety CI: Added ASAN (AddressSanitizer) workflow for nightly memory safety checks
  • Runs on schedule (daily) and manual dispatch
  • Validates no memory safety issues in core library
  • Local .cargo/check.sh --memory-checks option for developers with nightly toolchain
• Phase Reference Removal: Cleaned up implementation-plan-internal references from all user-facing code
  • Removed "Phase A-H" and gate nomenclature from source code comments
  • Removed from Python wrapper module documentation
  • Removed from test file names and test documentation
  • Removed from example code
  • Codebase now approachable to new users unfamiliar with development history
• Test File Reorganization:
  • test_h_gate_benchmarking.py → test_parallel_benchmarking.py
  • test_h5_integration.py → test_integration.py
  • test_queue_state_machine_c2.py → test_queue_state_machine.py
  • test_memory_profiling_c4.py → test_memory_profiling.py

Previously Added (2026-01-08)

Full pymarc API Parity ✅

• Leader Value Validation Helpers: Complete MARC 21 leader position reference implementation
  • Leader.get_valid_values(position) - Returns dict of valid values for each leader position
  • Leader.is_valid_value(position, value) - Validates values per MARC 21 specification
  • Leader.get_value_description(position, value) - Gets human-readable descriptions
  • Support for positions 5 (Record Status), 6 (Type of Record), 7 (Bibliographic Level), 17 (Encoding Level), 18 (Cataloging Form)
• Indicators as Tuple-Like Object: Full support for pymarc-compatible indicator access
  • field.indicators[0] and field.indicators[1] indexing
  • field.indicators unpacking support
  • Backward compatibility with field.indicator1 and field.indicator2 properties
• Control Field Access Pattern: Support for pymarc's record['001'].value pattern
  • ControlField wrapper with .value property for control fields (001-009)
  • Both record['001'].value and record.control_field('001') patterns work identically
  • Backward compatibility maintained

Documentation

• pymarc API Parity Plan: Moved completed work to docs/history/ for archival
• API Compatibility Summary: Historical record of all 7 API gaps identified and resolved
• README Enhancement: Updated to highlight full pymarc API compatibility as primary feature
• Migration Guide: Comprehensive guide showing all compatible patterns with minimal migration path

Changed

• README: Removed "nearly" qualifier - now describes "full" pymarc API compatibility
• Python Test Suite: Expanded from 88+ to comprehensive coverage of all parity features
• Documentation Structure: Organized completed parity work into history directory

Performance

• No performance regressions - all existing benchmarks maintained
• Leader validation methods are zero-copy (dictionary lookups only)

Technical Details

• MARC 21 Reference Data: All leader position value mappings per MARC 21 specification
• API Stability: Python wrapper API now guaranteed stable for pymarc compatibility
• Backward Compatibility: All new features fully backward compatible with existing code

[0.3.1] - 2026-01-07

Added

CI/CD Improvements

• Python Wheel Build Workflow: Fixed multi-platform wheel building for Ubuntu, macOS, and Windows
  • Configured maturin to use -i pythonX.Y flag for manylinux Python version selection (per maturin docs)
  • Fixed Windows glob expansion in test step using PowerShell conditionals
  • Removed deprecated mypy/pyright strict type checking (to be addressed in future typing effort)
  • Wheels now build and test successfully for Python 3.9, 3.10, 3.11, and 3.12 across all platforms

Changed

Documentation Updates

• Performance Reference Table (benchmarks/RESULTS.md): Updated to use pymarc as baseline (1.0x) for clearer speedup comparison
• Real-World Performance Scenarios: Standardized table formats across all four scenarios for consistent speedup and time-saved metrics
• PERFORMANCE.md Executive Summary: Updated to clarify recommended strategies (ProducerConsumerPipeline for single-file, ThreadPoolExecutor for multi-file)
• Documentation Audit: Moved completed audit to docs/history/ for archival

[0.3.0] - 2026-01-06

Added

GIL Release & Concurrency (Phase A-F)

• GIL Release during I/O: Python wrapper now releases GIL during record parsing for true multi-thread parallelism
• Three-Phase Pattern: Robust pattern separates Python object access (GIL held) from CPU-intensive parsing (GIL released)
• Measured Performance: 2.04x speedup on 2 threads, 3.20x on 4 threads (Phase H benchmarking)
• BatchedMarcReader: Queue-based buffering reduces GIL contention from N to N/100 operations
• SmallVec Optimization: 4 KB inline buffer avoids allocations for ~85-90% of MARC records

Parallel I/O Backend (Phase H)

• ReaderBackend Enum: Unified reader supporting multiple input types with automatic selection
  • RustFile: Pure Rust file I/O (zero GIL overhead)
  • CursorBackend: In-memory bytes (zero GIL overhead)
  • PythonFile: Python file objects (GIL-managed)
• File Path Support: Direct file path input bypasses Python I/O layer entirely
• Bytes/Bytearray Support: In-memory MARC data via CursorBackend
• Automatic Detection: Input type automatically detected, optimal backend selected

Documentation (Phase G)

• PERFORMANCE.md: Comprehensive guide with threading patterns, benchmarking methods, and tuning recommendations
• Threading Examples: Practical concurrent_reading.py and concurrent_writing.py examples
• API Documentation: Updated PyMarcReader/PyMarcWriter docstrings with threading guidance
• README Threading Section: Concrete speedup numbers (2.04x for 2 threads, 3.20x for 4)
• Benchmark Documentation: Updated docs/benchmarks/ with Phase H results

Changed

• Python Wrapper API: No breaking changes (fully backward compatible)
• Performance Profile: Single-thread throughput stable, multi-thread speedup now available
• Reader Construction: Accepts str/Path/bytes/bytearray in addition to file objects

Fixed

• GIL Release Bug (Phase B): SmallVec copy pattern properly avoids borrow checker violations
• Error Handling (Phase B): ParseError conversion happens after GIL re-acquisition
• GIL Crossing (Phase C): py.detach() correctly releases GIL during Phase 2 parsing

Performance

• Threading Speedup: 2.04x (2 threads), 3.20x (4 threads) vs sequential reading
• Memory Overhead: SmallVec buffering <3% overhead vs single-threaded
• GIL Contention: Reduced from linear to O(n/100) with BatchedMarcReader
• File I/O: Pure Rust backend (file paths) eliminates all GIL overhead

Technical Details

• Phase H Integration: Producer-consumer pipeline with Rayon parallel record scanning
• Backpressure: Queue-based buffering prevents runaway producer threads
• Thread Safety: Each thread requires own reader instance (not Send/Sync by design)
• Efficiency: pymrrc 92% efficient vs pure Rust Rayon baseline

[0.2.0] - 2025-12-31

Added

Python Integration (Phase 2-3)

• PyO3 Python Wrapper: Full Python bindings via PyO3/Maturin
• Python Parallel Benchmarks: Threading and multiprocessing performance analysis
• pymarc Compliance Test Suite: 75+ tests validating compatibility with pymarc API
• Memory Usage Profiling: Benchmarks comparing Python wrapper overhead vs Rust native
• Context Manager Support: Python with statement support for file I/O

Performance & Benchmarking (Phase 1-4)

• Rust Parallel Benchmarks: Performance testing with rayon for concurrent MARC processing
• Rayon Parser Pool (H.4b): Parallel batch processing functions via Rayon thread pool
  • parse_batch_parallel() - Unlimited parallel parsing with dynamic work distribution
  • parse_batch_parallel_limited() - Bounded parallel parsing respecting configured thread pool
  • Record boundary scanning and parallel record assembly
  • Thread-safe batch processing for large MARC files
• Comprehensive Benchmark Suite:
  • 1K/10K/100K record read performance
  • Field access overhead measurements
  • JSON/XML serialization performance
  • Roundtrip (read+write) benchmarks
  • Sequential vs parallel processing comparison
• Benchmark Results Documentation: Real measured performance data with pymarc comparisons
• CI Optimization: Caching and performance gate integration

Documentation & Organization

• Design Documentation: Architecture and design decision documentation
• Benchmarking Documentation: Feasibility studies and performance analysis
• Executive Summaries: High-level overview of parallel processing capabilities
• Reorganized Docs: Consolidated design/ and history/ into docs/ hierarchy
• Examples: Real-world usage patterns demonstrating library features
• Migration Guide: Comprehensive guide highlighting near-100% API compatibility with pymarc

API Enhancements

• Record.to_marc21(): Convert records back to ISO 2709 binary format
• Enhanced Python API: Full feature parity with Rust API in Python wrapper
• MARCWriter Fixes: Fixed write() method and improved record serialization

Fixed

• Fixed deprecated PyO3 type alias warning
• Fixed 20+ clippy linting violations in benchmark files (missing semicolons in closure statements)
• Suppressed benchmark-specific documentation warnings at file level
• Fixed MARCWriter record serialization issues
• Cleaned up CI pipeline to pass all quality gates
• H.4a Bug Fix: RecordBoundaryScanner now correctly scans for 0x1D (record terminator) instead of 0x1E (field terminator) per ISO 2709 specification

Changed

• Improved documentation structure and organization
• Enhanced Python test coverage with comprehensive pymarc compatibility suite

[0.1.0] - 2025-12-28

Added

Core Features

• ISO 2709 Binary Format: Full read/write support for MARC records in the standard binary interchange format
• Record Types: Support for three MARC record types:
  • Bibliographic records (standard MARC records)
  • Authority records (Type Z) for standardized headings and cross-references
  • Holdings records (Types x/y/v/u) for item location and enumeration data
• Builder Pattern API: Fluent, idiomatic Rust interface for record construction
• Field Access API: Comprehensive methods for reading, filtering, and iterating over fields
  • fields_by_tag() - Get fields by tag
  • fields_by_indicator() - Filter by indicators
  • fields_in_range() - Get fields within a tag range
  • fields_with_subfield() - Get fields containing specific subfields

Advanced Query DSL (Phase 1-3)

• FieldQuery Builder: Complex criteria-based field matching (tags, indicators, subfields)
• TagRangeQuery: Range-based field lookups (e.g., 600-699)
• Subfield Pattern Matching: Regex-based field filtering
• Linked Field Navigation: Support for MARC 880 (Alternate Graphical Representation) fields
  • Parse linkage information from subfield 6
  • Bidirectional lookups between original and 880 fields
• Authority Control Helpers: Query traits for authority-specific operations
• Format-Specific Traits:
  • BibliographicQueries for bibliographic records
  • AuthorityQueries for authority records
  • HoldingsQueries for holdings records

Serialization Formats

• JSON: Generic JSON representation with fields as keys
• MARCJSON: Standard JSON-LD format for MARC records
• XML: XML representation with proper field/subfield structure
• CSV: Tabular export format for spreadsheet applications
• Dublin Core: Simplified 15-element metadata schema
• MODS: Metadata Object Description Schema for detailed descriptions

Character Encoding Support

• MARC-8 (Legacy): Full support for legacy MARC-8 encoding with:
  • Basic Latin (ASCII)
  • ANSEL Extended Latin with diacritical marks
  • Hebrew (ESC ) 2)
  • Arabic (ESC ) 3, 4)
  • Cyrillic (ESC ( N)
  • Greek (ESC ( S)
  • Subscripts/Superscripts/Special character sets
  • East Asian (Chinese, Japanese, Korean via EACC)
• UTF-8 (Modern): Full Unicode support for modern MARC records
• Automatic Detection: Encoding detection from MARC leader position 9

Helper Methods

• RecordHelpers Trait: Available on all record types via blanket implementation
  • title() - Extract main title
  • author() / authors() - Extract author names
  • subjects() - Extract subject headings
  • isbns() - Extract ISBN values
  • issns() - Extract ISSN values
  • publication_info() - Extract publication details
  • Record type helpers: is_book(), is_music(), is_map(), is_serial(), is_audiovisual(), is_electronic_resource()

API Refactoring

• MarcRecord Trait: Common interface for all record types (control field operations)
• GenericRecordBuilder: Unified builder for all record types
• FieldCollection Trait: Standardized field collection management
• Unified field storage pattern across Record, AuthorityRecord, and HoldingsRecord

Error Handling & Recovery

• MarcError Type: Comprehensive error handling for MARC operations
• Recovery Mode: Graceful handling of truncated/malformed MARC records
• Result Type: Convenient Result<T> alias for library operations

Testing

• 282+ comprehensive unit and integration tests
• Test data with:
  • Simple bibliographic records
  • Music scores
  • Records with control fields (008)
  • Multiple records in one file
  • Authority records
  • Holdings records
  • Multilingual records
  • MARC-8 encoded records

Documentation

• Comprehensive API documentation with doc comments
• Module-level documentation with examples
• Examples directory with real-world usage patterns:
  • Creating records with builders
  • Reading and querying fields
  • Converting between formats
  • Working with authority and holdings records
  • MARC-8 encoding demonstration
  • Multilingual record handling
  • CSV export

Design Principles

1. Rust-Idiomatic: Leverages iterators, Result types, and ownership patterns naturally
2. Zero-Copy Where Possible: Efficient memory usage for large record sets
3. Format Flexibility: Support for multiple serialization formats out of the box
4. Compatibility: Maintains data fidelity with pymarc and standard MARC tools
5. Extensible: Trait-based architecture allows easy addition of new query types and formats

Known Limitations

None known at this time. The following have been resolved:

• ✓ Field indicator validation with MARC21 semantics (implemented in 0.1.0)
• ✓ MARC-8 combining character handling with Unicode NFC normalization (implemented in 0.1.0)

Technical Details

• Minimum Rust Version: 1.70
• Dependencies: serde, serde_json, regex, unicode-normalization
• License: MIT

Project Information

• Repository: https://github.com/dchud/mrrc
• Documentation: https://docs.rs/mrrc
• Crates.io: https://crates.io/crates/mrrc
• Issue Tracking: GitHub Issues and Beads issue tracking system
• Status: Active development, experimental (APIs may change)