Skip to content

feat(snowflake): transpile CORR with NaN-->NULL#6619

Merged
treysp merged 2 commits intomainfrom
trey/corr-sf-duckdb
Jan 7, 2026
Merged

feat(snowflake): transpile CORR with NaN-->NULL#6619
treysp merged 2 commits intomainfrom
trey/corr-sf-duckdb

Conversation

@treysp
Copy link
Collaborator

@treysp treysp commented Dec 22, 2025

CORR performs an internal calculation that has VARIANCE([column]) in the denominator. When that occurs, Snowflake returns NULL while DuckDB returns NaN.

To replicate the Snowflake's NULL in DuckDB, we must wrap the CORR call in ISNAN:

CASE WHEN ISNAN(CORR(a, b)) THEN NULL ELSE CORR(a, b) END

Naively wrapping in ISNAN generates invalid syntax for windowed and filtered function calls.

Therefore, we add special DuckDB generator handling to route exp.Window, exp.Window(exp.Filter), and exp.Filter to a corr_sql method designed to accept window and filtered expressions.

Examples:

  • Snowflake: SELECT CORR(a, b) OVER (PARTITION BY c)
    • DuckDB: SELECT CASE WHEN ISNAN(CORR(a, b) OVER (PARTITION BY c)) THEN NULL ELSE CORR(a, b) OVER (PARTITION BY c) END
  • Snowflake: SELECT CORR(a, b) FILTER (WHERE c > 0)
    • DuckDB: SELECT CASE WHEN ISNAN(CORR(a, b) FILTER(WHERE c > 0)) THEN NULL ELSE CORR(a, b) FILTER(WHERE c > 0) END
  • Snowflake: SELECT CORR(a, b) FILTER (WHERE c > 0) OVER (PARTITION BY d)
    • DuckDB: SELECT CASE WHEN ISNAN(CORR(a, b) FILTER(WHERE c > 0) OVER (PARTITION BY d)) THEN NULL ELSE CORR(a, b) FILTER(WHERE c > 0) OVER (PARTITION BY d) END

@treysp treysp requested a review from georgesittas December 22, 2025 22:44
@github-actions
Copy link
Contributor

github-actions bot commented Dec 22, 2025

SQLGlot Integration Test Results

Comparing:

  • this branch (sqlglot:trey/corr-sf-duckdb, sqlglot version: trey/corr-sf-duckdb)
  • baseline (main, sqlglot version: 28.5.1.dev72)

⚠️ Limited to dialects: snowflake, duckdb

By Dialect

dialect main sqlglot:trey/corr-sf-duckdb difference links
duckdb -> duckdb 4003/4003 passed (100.0%) 4003/4003 passed (100.0%) No change full result / delta
snowflake -> duckdb 578/847 passed (68.2%) 578/847 passed (68.2%) No change full result / delta
snowflake -> snowflake 847/847 passed (100.0%) 847/847 passed (100.0%) No change full result / delta

Overall

main: 5697 total, 5428 passed (pass rate: 95.3%), sqlglot version: 28.5.1.dev72

sqlglot:trey/corr-sf-duckdb: 5697 total, 5428 passed (pass rate: 95.3%), sqlglot version: trey/corr-sf-duckdb

Difference: No change

@treysp treysp force-pushed the trey/corr-sf-duckdb branch from 65a57ea to 0d175bc Compare December 29, 2025 23:04
@georgesittas
Copy link
Collaborator

@treysp let me know if there's anything blocking this.

@treysp treysp force-pushed the trey/corr-sf-duckdb branch from 0d175bc to 2323b2d Compare January 7, 2026 17:51
@treysp treysp marked this pull request as ready for review January 7, 2026 17:51
@treysp treysp merged commit 26c16f2 into main Jan 7, 2026
9 checks passed
@treysp treysp deleted the trey/corr-sf-duckdb branch January 7, 2026 18:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants