parquet_indexer (Spotlight Importer for .parquet)

Warning

This repository was written completely using Codex. Model: GPT-5 (Codex coding agent) Version: Codex GPT-5 session build (2026-03-12) It works for me, but I take no guarantee if it works for you.

This project provides:

• a macOS Spotlight importer (.mdimporter) for Parquet metadata indexing
• a modern Quick Look preview extension (.appex, embedded in a host app) for Parquet metadata preview with:
  • hierarchical schema tree (collapsible groups)
  • full parsed column list (no preview truncation)
  • Arrow-style logical scalar type labels where available
• a small GUI manager app (Parquet Quick Look and Index.app) with:
  • Re-Install, Repair Registration, and Uninstall actions
  • Copy Diagnostic Report action for support/issue reports (anonymized)
  • Quick Look rendering settings (schema display + dataset scan controls)
  • dedicated Updates tab for release checks (Daily/Weekly/Monthly)
  • automatic importer install on app launch if missing

Since 0.3.0, naming is standardized to Apple's official Quick Look terminology.

The importer reads only Parquet footer metadata (...<metadata_len><PAR1>), does not parse row contents, and writes Spotlight metadata for search.

Prerequisites

• Minimum supported macOS version: 12.0 (Monterey).

• macOS with Spotlight enabled

• Xcode Command Line Tools (xcrun, clang, codesign)

• shell access (zsh)

• permission to write to ~/Library/Spotlight and ~/Applications

Optional checks:

xcrun --show-sdk-path
clang --version
codesign --version

Build

From the project root:

./scripts/build.sh

This creates:

• build/Parquet.mdimporter
• build/Parquet Quick Look and Index.app (contains ParquetQuickLook.appex)

The build script also:

• copies Info.plist and resources/schema.xml
• clears extended attributes
• signs the bundle ad-hoc (required for reliable loading)

Release Packaging

Automate build + DMG + GitHub release upload:

./scripts/release.sh v0.4.2

What it does:

• runs ./scripts/generate_icon.sh and ./scripts/build.sh
• creates release/Parquet-Spotlight-0.4.2.dmg
• creates release/Parquet-Spotlight-0.4.2.zip
• creates release/Parquet-Spotlight-0.4.2.dmg.sha256
• creates or updates the GitHub release tag and uploads those assets
• verifies release preflight before packaging:
  • current branch is main
  • working tree is clean
  • local main is fully synced with upstream

Optional flags:

• --notes <file>: use a custom release notes file
• --skip-gh: package artifacts only, do not call GitHub CLI

GitHub Release Install (DMG)

Download the latest DMG from:

• GitHub Releases

Typical asset name:

• Parquet-Spotlight-<version>.dmg

Unsigned app note (important):

• Releases are currently unsigned (ad-hoc signed for local loading, but not Apple Developer ID notarized).
• On first launch, macOS may block the app with a security warning.
• If blocked, right-click the app and choose Open, or go to:
  • System Settings -> Privacy & Security -> Open Anyway

Rollback (Revert To A Previous Version)

If a new release causes problems:

1. Uninstall current version (recommended):

./scripts/uninstall.sh

2. Download an older DMG from:
   • GitHub Releases
3. Install that older app and run Re-Install (or Repair Registration) once.

Optional verification after rollback:

mdimport -t -d2 /path/to/file.parquet
qlmanage -p /path/to/file.parquet

Install

Install for current user:

./scripts/install.sh

This installs and registers:

• ~/Library/Spotlight/Parquet.mdimporter
• ~/Applications/Parquet Quick Look and Index.app (embedded modern Quick Look extension)

Use Parquet Quick Look and Index.app from ~/Applications to:

• auto-install importer payload if missing (on launch)
• re-install importer payload from inside the app bundle
• repair registration and refresh caches
• uninstall importer + Quick Look app (including settings cleanup)
• change preview settings used by Quick Look
• check for updates (menu item + Updates tab), with Skip this version option

Uninstall

Remove importer and preview for current user:

./scripts/uninstall.sh

This performs full user-level cleanup:

• app/importer/plugin removal
• Quick Look cache refresh + Finder restart
• settings/preferences/cache cleanup (best effort for container root metadata)

Dry-run mode:

./scripts/uninstall.sh --dry-run

Test

1. Test-import a file and confirm plugin selection:

mdimport -t -d2 /path/to/file.parquet

Expected output includes:

• with plugIn /Users/<you>/Library/Spotlight/Parquet.mdimporter

2. Inspect indexed metadata for one file:

mdls -name kMDItemKind \
     -name com_rkrug_parquet_is_valid \
     -name com_rkrug_parquet_file_size \
     -name com_rkrug_parquet_footer_length \
     -name com_rkrug_parquet_row_count \
     -name com_rkrug_parquet_column_count \
     /path/to/file.parquet

3. Reindex if needed:

mdimport -r ~/Library/Spotlight/Parquet.mdimporter
mdimport /path/to/file.parquet

3.1. Reload Quick Look preview plugins if needed:

qlmanage -r
qlmanage -r cache

4. Run installation self-test:

./scripts/test_install.sh

Optional explicit test file:

./scripts/test_install.sh /path/to/file.parquet

5. Test Quick Look preview plugin:

qlmanage -p /path/to/file.parquet

6. Test dataset folder preview:

qlmanage -p /path/to/parquet-dataset-folder

Behavior:

• folders containing parquet files render a dataset summary preview
• non-parquet folders fall back to standard macOS folder Quick Look

Dataset scan behavior is controlled in app settings:

• Scan all files
• Max files (if not all)
• Recursive scan folders

7. Run script + core logic test suite:

./scripts/test_scripts.sh

This also runs:

./scripts/test_core.sh

test_core.sh validates trusted update URL/tag checks, version comparisons, error mapping, and diagnostics redaction rules.

Search

Spotlight GUI

Use:

• kind:Apache Parquet file
• name:.parquet

Terminal (mdfind)

mdfind 'kMDItemKind == "Apache Parquet file"'
mdfind 'com_rkrug_parquet_is_valid == 1'
mdfind 'com_rkrug_parquet_footer_length > 100000'
mdfind 'com_rkrug_parquet_file_size > 100000000'
mdfind 'com_rkrug_parquet_row_count > 1000000'
mdfind 'com_rkrug_parquet_column_count > 100'
mdfind 'com_rkrug_parquet_columns == "species"'
mdfind 'kMDItemKeywords == "col-species"'

Troubleshooting

mdimport says with no plugIn

Run:

mdimport -r ~/Library/Spotlight/Parquet.mdimporter
mdimport -t -d2 /path/to/file.parquet

If needed, reinstall:

./scripts/build.sh
./scripts/install.sh

Custom fields are null in mdls

Force reindex the file:

mdimport /path/to/file.parquet

Then check again:

mdls -name com_rkrug_parquet_is_valid \
     -name com_rkrug_parquet_footer_length \
     -name com_rkrug_parquet_file_size \
     /path/to/file.parquet

Spotlight results seem stale

Re-register plugin and reindex a folder:

mdimport -r ~/Library/Spotlight/Parquet.mdimporter
mdimport /path/to/folder

Preview plugin not listed in qlmanage -m plugins

qlmanage -m plugins primarily lists legacy generators and may not reflect modern preview extensions.

Use this instead:

pluginkit -m -p com.apple.quicklook.preview | rg parquet

Spotlight indexing via .mdimporter is independent and unaffected.

Exposed Properties

Standard Spotlight fields:

• kMDItemTitle
• kMDItemKind (Apache Parquet file)
• kMDItemDescription
• kMDItemKeywords
• kMDItemTextContent (metadata tokens only; no row text)

Custom Parquet fields:

• com_rkrug_parquet_is_valid (CFBoolean)
• com_rkrug_parquet_file_size (CFNumber)
• com_rkrug_parquet_footer_length (CFNumber)
• com_rkrug_parquet_row_count (CFNumber)
• com_rkrug_parquet_column_count (CFNumber)
• com_rkrug_parquet_columns (CFString, multivalued)

Queryable mdfind Properties

• com_rkrug_parquet_is_valid (CFBoolean):

  • 1 if the file has a readable Parquet trailer/footer (PAR1 + footer length), else 0.
  • Example: mdfind 'com_rkrug_parquet_is_valid == 1'

• com_rkrug_parquet_file_size (CFNumber):

  • Total parquet file size in bytes (from filesystem stat at import time).
  • Example: mdfind 'com_rkrug_parquet_file_size > 100000000'

• com_rkrug_parquet_footer_length (CFNumber):

  • Footer metadata length in bytes (value from Parquet trailer).
  • Example: mdfind 'com_rkrug_parquet_footer_length > 50000'

• com_rkrug_parquet_row_count (CFNumber):

  • Best-effort extracted row count from footer metadata.
  • Example: mdfind 'com_rkrug_parquet_row_count > 1000000'

• com_rkrug_parquet_column_count (CFNumber):

  • Number of extracted columns stored in com_rkrug_parquet_columns.
  • Example: mdfind 'com_rkrug_parquet_column_count > 100'

• com_rkrug_parquet_columns (CFString, multivalued):

  • Best-effort extracted column names from footer metadata.
  • Example: mdfind 'com_rkrug_parquet_columns == "Species"'

• kMDItemKeywords (CFString, multivalued):

  • Includes generated tokens such as col-<column_name>, parquet-valid, parquet-footer-<N>.
  • Example: mdfind 'kMDItemKeywords == "col-species"'

Keyword tokens written by importer:

• parquet
• parquet-valid / parquet-invalid
• parquet-footer-<N>
• parquet-size-<N>
• parquet-rows-<N>
• parquet-cols-<N>
• col-<column_name>

Notes

• UTI: com.rkrug.parquet
• Bundle ID: com.rkrug.parquetindexer.importer
• Plugin factory UUID: 0E198062-E6D8-4AC2-BBCE-FB860A43A116
• Preview extension bundle ID: com.rkrug.parquetindexer.previewhost.preview
• Quick Look host app bundle ID: com.rkrug.parquetindexer.previewhost

License

This project is licensed under the MIT License. See LICENSE.