Skip to content

Commit 71ea07f

Browse files
MaxHalfordclaude
andauthored
Quack mode (#174)
Solves #32. Blog post incoming! I also introduced a `.lea_cache` to cache the SQLGlot tokenization process (~6s for Carbonfact's scripts). It's what SQLMesh does too. This combined with deferring heavy unnecessary imports makes the CLI feel instant. And I moved from pre-commit to prek and fixed a lot of code quality issues. <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Adds Quack mode to run most of the DAG locally on DuckLake with automatic SQL transpilation, only using the native warehouse (e.g. BigQuery) for scripts with external dependencies. Also adds a `.lea_cache` to cache SQLGlot parsing for faster DAG builds and switches tooling to `prek`. - New Features - Quack mode (`--quack`): classifies scripts, runs eligible ones on DuckLake, and transpiles SQL via `sqlglot`; only external-dependent scripts (and their ancestors) run on the native warehouse. - Added `DuckLake` warehouse support, dialect attach/setup, and robust transpilation/classification logic with extensive tests. - Script parsing cache (`.lea_cache`) to store dependencies; includes tests and a `examples/quack` demo. - Dependencies - Replace pre-commit with `prek`; CI updated to `uv run prek run --all-files`. Add `ruff` and `ty` hooks. - Bump to version 0.17.0 and relax `sqlglot` pin; minor README updates. <sup>Written for commit d5414d6. Summary will update on new commits.</sup> <!-- End of auto-generated description by cubic. --> --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent 04a9b98 commit 71ea07f

29 files changed

+2738
-510
lines changed

.github/workflows/ci.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,5 +18,5 @@ jobs:
1818
- uses: astral-sh/setup-uv@v7
1919
with:
2020
enable-cache: true
21-
- run: uv run pre-commit run --all-files
21+
- run: uv run prek run --all-files
2222
- run: uv run pytest

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,6 @@ dist/
66
/*.ipynb
77
.DS_Store
88
*.wal
9+
*.parquet
10+
*.ducklake
11+
ui

.pre-commit-config.yaml

Lines changed: 15 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,19 @@
1-
files: .
21
repos:
3-
- repo: https://github.com/pre-commit/pre-commit-hooks
4-
rev: v4.4.0
5-
hooks:
6-
- id: check-json
7-
- id: check-yaml
8-
- id: trailing-whitespace
9-
- id: mixed-line-ending
10-
11-
- repo: https://github.com/astral-sh/ruff-pre-commit
12-
rev: v0.1.7
2+
- repo: local
133
hooks:
144
- id: ruff
5+
name: ruff linter
6+
entry: uv run ruff check --fix
7+
language: system
8+
types: [python]
159
- id: ruff-format
16-
- repo: local
17-
hooks:
18-
- id: check-external-actions-pinned
19-
name: Check GitHub Actions are pinned
20-
entry: pre-commit-hooks/check_pinned_actions.sh
21-
language: script
22-
pass_filenames: false
10+
name: ruff formatter
11+
entry: uv run ruff format
12+
language: system
13+
types: [python]
14+
- id: ty
15+
name: ty type checker
16+
entry: uv run ty check lea
17+
language: system
18+
types: [python]
19+
pass_filenames: false

0 commit comments

Comments
 (0)