Conversation
macOS-13 (Intel) runners are deprecated. Use macos-latest (Apple Silicon) only. Remove TestPyPI publishing as it's not needed.
Major version upgrade from Tika 2.9.2 (EOL) to Tika 3.3.0 with GraalVM CE 25.0.2 and Gradle 9.2.0. Based on work proven by lamco-admin and a12591771 community forks. Build changes: - Tika 3.3.0, GraalVM plugin 0.10.4, slf4j 2.0.16, log4j 2.24.2 - Add jakarta.mail and angus-mail deps (Tika 3 javax->jakarta migration) - Remove tika-parser-html-commons (merged into html-module in Tika 3) - GraalVM 25 build flags: RemoveUnusedSymbols, ReportExceptionStackTraces - Drop macOS x86_64 (GraalVM 25 removed Intel Mac support) - Add *.xsb glob to reachability metadata for poi-ooxml-full schemas Code changes: - Fix BodyContentHandler API (Tika 3 removed OutputStream constructor) - Fix UTF-8 chunk boundary bug in Python test stream reader Test updates: - Remove access_permission:can_print_degraded from PDF expected metadata - Update PNG Content-Type from image/png to image/ocr-png
Move PNG and OCR-dependent tests out of parametrized suites into dedicated #[ignore] tests. These require Tesseract and timeout in constrained environments. Run with `cargo test -- --ignored` in CI.
Tika 3.3.0's EpubParser.bufferedParseZipFile concatenates the OPF's parent directory with each manifest href and looks up the literal string via commons-compress ZipFile.getEntry, which does not collapse ".." segments. EPUBs whose OPF references an entry at the archive root (e.g. href="../toc.xhtml") trip the strict content-item check, throw EpubZipException, and surface to callers as "TIKA-198: Illegal IOException from EpubParser". parseFileToString and parseBytesToString now detect this case and retry via a lenient fallback that opens the zip directly, parses the OPF for spine order + Dublin Core metadata, and runs AutoDetectParser per spine entry inside an EmbeddedContentHandler. Entry paths are normalized via Paths.get(...).normalize() so ".." traversals resolve. Individual entry failures are recorded as X-TIKA:warning metadata but do not abort the overall parse. Healthy EPUBs keep using the stock path unchanged. See docs/patches.md for full root-cause analysis and upstream cleanup checklist. The reader-based API (parseFile / parseUrl) is not yet covered.
Switch the base image to mcr.microsoft.com/devcontainers/javascript-node and move NODE_OPTIONS from containerEnv into ~/.zshrc. Zed's devcontainer env parser chokes when Config.Env values contain "=" (as NODE_OPTIONS does with "--max-old-space-size=4096"), so exporting it from the shell rc file keeps the same value available inside the container without breaking Zed.
Tika's TIKA-216 SecureContentHandler throws a SAXException when XML element nesting exceeds 100 levels, which misfires on legitimate content like verse poetry typeset as one <div> per line. The failure propagates as TIKA-237 and aborts the whole EPUB parse. isEpubLenientCandidate now retries TIKA-237 via lenientEpubParseToString when the file is an EPUB-family container (application/epub+zip or application/x-ibooks+zip) and rootMessage(e) contains "zip bomb". The content-type fallback is needed because the TIKA-237 wrapper surfaces the outer composite parser rather than EpubParser, and the zip-bomb gating keeps unrelated SAX failures (malformed XHTML, broken entities) surfacing as hard errors instead of being silently salvaged. Reproducer: cauldron/error-files/tika-237-saxexception_9791220847322.epub (Beowulf with 258-level nested div verse in chapter 3).
Two related bugs caused ~/.gitconfig inside the container to lose the host identity and credential helper overrides after any image rebuild: 1. Dockerfile wrote a baseline ~/.gitconfig at image build time via `git config --global`. That file landed on a lower overlayfs layer and survived subsequent container starts. setup-gitconfig.sh was supposed to override it from postCreateCommand, but postCreateCommand only runs on fresh container create — not on rebuilds — so any rebuild silently reset the config. 2. All four .devcontainer/*.sh scripts were tracked in git as 100644 (non-executable). postCreateCommand invokes them as bare paths (`.devcontainer/setup-gitconfig.sh && ...`), which fails with "permission denied" on a fresh checkout. Even when postCreateCommand did run, the scripts themselves silently did nothing. Drop the redundant `git config --global` block from the Dockerfile and chmod +x the four devcontainer setup scripts in git's index.
Exposes extract_file_metadata, extract_bytes_metadata, and extract_url_metadata on the core Extractor and Python bindings for workflows that need document metadata but not the full text. Skipping content extraction is typically 2–5× faster on text-heavy PDFs because PDFBox glyph rendering is bypassed.
Flattens to the iscc-sdk bullet style (no Bug Fixes/Development subheadings), moves all pre-release work into a 0.4.0 - 2026-03-29 section, and records the post-0.4.0 entries on develop: metadata-only extraction API, Tika 3.3.0 upgrade, TIKA-198 and TIKA-237 EPUB fallbacks, OCR test ignore, devcontainer fixes, and README update. Drops the broken auto-generated 0.1.x entries in favor of a link to the upstream extractous changelog.
- README.md: re-wrap metadata-API paragraph at mdformat's 100-char width. - docs/patches.md: convert the Applied/Affects/Scope/Upstream header block into a bullet list so mdformat no longer joins the four lines into one run-on paragraph, and re-wrap the rest of the file. - extractor.rs, extract_to_string_tests.rs: apply cargo-fmt to drift left over from the metadata-only API commit. - errors.rs: delete. The module has been an empty placeholder (single newline) since the project's inception and nothing imports from it — lib.rs only had `mod errors;` with the `pub use` line commented out. Removing it also breaks a deadlock between end-of-file-fixer (wants an empty file) and cargo-fmt (wants a trailing newline).
Silences the deprecation warnings about actions running on Node.js 20, which will be forced to Node.js 24 starting June 2, 2026 and removed from runners on September 16, 2026. - actions/checkout: v4 -> v6 (Node 24 since v5, latest v6.0.2) - actions/setup-python: v5 -> v6 (Node 24 since v6, latest v6.2.0) - actions/upload-artifact: v4 -> v7 (Node 24 since v5, latest v7.0.1) - actions/download-artifact: v4 -> v8 (Node 24 since v6, latest v8.0.1) No breaking usages: all artifact operations are name/pattern based (not artifact-id based), uploads are standard zipped directory uploads (not the new opt-in direct-upload mode), and GitHub-hosted runners always meet the minimum runner versions these actions require.
titusz
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tika-3.3.0-*dirs,*.xsbglob for poi-ooxml-full). Replaces Tika 2.9.2 (EOL April 2025).extract_file_metadata,extract_bytes_metadata,extract_url_metadataon both RustExtractorand the Python bindings. Typically 2–5× faster on text-heavy PDFs by bypassing PDFBox glyph rendering.TIKA-198(EpubZipExceptionfrom unnormalized..path segments) andTIKA-237(false-positive zip-bomb detection on deeply nested<div>content) by retrying spine parsing via a freshAutoDetectParser. Records a warning inX-TIKA:warning. Seedocs/patches.mdfor the root-cause analysis. Reader-based API (extract_file/extract_bytes) is not yet covered.#[ignore]for fast local runs (cargo test -- --ignoredfor the full suite).NODE_OPTIONShandling for Zed.Test plan
cargo build/cargo test/cargo clippy -D warningsgreen locallymaturin develop -E test+pytestgreen locally