Skip to content

fix: Skip statistics deserialization for Null columns in Parquet#27021

Draft
0xRozier wants to merge 6 commits intopola-rs:mainfrom
0xRozier:fix/predicate-pushdown-null-column-stats-26974
Draft

fix: Skip statistics deserialization for Null columns in Parquet#27021
0xRozier wants to merge 6 commits intopola-rs:mainfrom
0xRozier:fix/predicate-pushdown-null-column-stats-26974

Conversation

@0xRozier
Copy link
Contributor

Fixes #26974

Return Ok(None) early for D::Null in deserialize_all, consistent with Dictionary, FixedSizeList, and Struct. This prevents a ShapeError caused by incomplete statistics arrays when predicate pushdown is used on a Null column with a schema override.

No use of AI

@github-actions github-actions bot added A-io-parquet Area: reading/writing Parquet files fix Bug fix python Related to Python Polars rust Related to Rust Polars labels Mar 24, 2026
@0xRozier 0xRozier marked this pull request as draft March 24, 2026 14:07
@codecov
Copy link

codecov bot commented Mar 24, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.39%. Comparing base (4cd2447) to head (45b8ab6).
⚠️ Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #27021      +/-   ##
==========================================
+ Coverage   80.92%   81.39%   +0.46%     
==========================================
  Files        1809     1809              
  Lines      249500   249497       -3     
  Branches     3140     3140              
==========================================
+ Hits       201919   203078    +1159     
+ Misses      46776    45614    -1162     
  Partials      805      805              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-io-parquet Area: reading/writing Parquet files fix Bug fix python Related to Python Polars rust Related to Rust Polars

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Predicate pushdown results in ShapeError

1 participant