-
Notifications
You must be signed in to change notification settings - Fork 77
feat: reworked build process and package structure #131
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
jurerotar
wants to merge
21
commits into
sqlite:main
Choose a base branch
from
jurerotar:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
21 commits
Select commit
Hold shift + click to select a range
0461194
feat: reworked build process and package structure (#3)
jurerotar bfcecac
fix: added sqlite3-opfs-async-proxy.js & sqlite3-worker1.mjs to /dist
jurerotar 9f88ba4
chore: update SQLite Wasm binaries from master (9ab2a2af1bb1c7d9777b1…
github-actions[bot] 893cb17
chore: update SQLite Wasm binaries from master (428c2656ff1b0023d06b6…
github-actions[bot] 0f82b33
chore: updated rolldown outputOptions.legalComments to preserve licen…
jurerotar ee970a6
chore: removed fetch patching
jurerotar 217ccca
chore: update SQLite Wasm binaries from master (bc2101b44ea60a737a591…
github-actions[bot] 498f4b4
chore: update SQLite Wasm binaries from master (35b9709d4675eb1ff9a8d…
github-actions[bot] 9a4b02f
chore: added wasm-strip dependency to dockerfile
jurerotar e91ff4b
Merge remote-tracking branch 'origin/main'
jurerotar 69eeb77
fix: fixed fetching sqlite by commit hash
jurerotar 4e5150f
chore: updated GitHub actions to make node tests run against all supp…
jurerotar 9b9df71
chore: cached Playwright browsers to prevent constant re-download
jurerotar 64faac0
chore: update SQLite Wasm binaries from master (43eb1d4d22f1644a4fd05…
github-actions[bot] 5ea1b51
chore: update SQLite Wasm binaries from master (90942efd218fe3e0a46e9…
github-actions[bot] 8ab42f6
chore: update SQLite Wasm binaries from master (4b9def19035dfabe6272c…
github-actions[bot] 77eefcb
Merge remote-tracking branch 'origin/chore/refactored-types'
jurerotar 63a481a
Merge branch 'sqlite:main' into main
jurerotar 70c45a9
chore: update SQLite Wasm binaries from master (8c4e315e19a36afc85e01…
github-actions[bot] b47a8a2
chore: removed version checks from tests
jurerotar 40918ef
chore: update .github/workflows/build-wasm.yml
jurerotar File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| root = true | ||
|
|
||
| [*] | ||
| ij_html_space_inside_empty_tag = true | ||
| end_of_line = lf | ||
| insert_final_newline = true | ||
| charset = utf-8 | ||
| indent_style = space | ||
| indent_size = 2 | ||
| trim_trailing_whitespace = true | ||
|
|
||
| # quote style | ||
| ij_javascript_force_quote_style = true | ||
| ij_typescript_force_quote_style = true | ||
| ij_javascript_use_double_quotes = false | ||
| ij_typescript_use_double_quotes = false | ||
|
|
||
| # bracket spacing | ||
| ij_javascript_spaces_within_object_literal_braces = true | ||
| ij_typescript_spaces_within_object_literal_braces = true | ||
| ij_javascript_spaces_within_object_type_braces = true | ||
| ij_typescript_spaces_within_object_type_braces = true | ||
|
|
||
| # imports | ||
| ij_javascript_spaces_within_imports = true | ||
| ij_typescript_spaces_within_imports = true | ||
| ij_javascript_import_merge_members = true | ||
| ij_typescript_import_merge_members = true | ||
| ij_javascript_import_sort_members = true | ||
| ij_typescript_import_sort_members = true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| inputs: | ||
| node-version: | ||
| description: 'Node.js version' | ||
| required: false | ||
| default: 'lts/*' | ||
|
|
||
| runs: | ||
| using: 'composite' | ||
| steps: | ||
| - name: Setup Node.js | ||
| uses: actions/setup-node@v6 | ||
| with: | ||
| node-version: ${{ inputs.node-version }} | ||
| cache: 'npm' | ||
| cache-dependency-path: package-lock.json | ||
|
|
||
| - name: Install dependencies | ||
| shell: bash | ||
| run: | | ||
| set -euo pipefail | ||
| npm ci --prefer-offline --no-audit --no-fund |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,119 @@ | ||
| name: Build SQLite Wasm | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| inputs: | ||
| sqlite_ref: | ||
| description: 'SQLite reference (tag, branch, or commit, or "latest")' | ||
| required: true | ||
| default: 'master' | ||
|
|
||
| jobs: | ||
| build: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| persist-credentials: false | ||
|
|
||
| - name: Resolve SQLite reference | ||
| id: resolve-ref | ||
| run: | | ||
| SQLITE_REF="${{ github.event.inputs.sqlite_ref }}" | ||
| if [ "$SQLITE_REF" = "latest" ]; then | ||
| echo "Fetching latest tag..." | ||
| LATEST_TAG=$(git ls-remote --tags --sort="v:refname" https://github.com/sqlite/sqlite.git "refs/tags/version-*" | tail -n 1 | cut -f 2 | sed 's/refs\/tags\///') | ||
| echo "Latest tag found: $LATEST_TAG" | ||
| SQLITE_REF="$LATEST_TAG" | ||
| fi | ||
|
|
||
| # Get the full commit SHA | ||
| SQLITE_SHA=$(git ls-remote https://github.com/sqlite/sqlite.git "$SQLITE_REF" | head -n 1 | cut -f 1) | ||
| if [ -z "$SQLITE_SHA" ]; then | ||
| # If not found, maybe it's a tag that needs refs/tags/ prefix or it's already a SHA | ||
| SQLITE_SHA=$(git ls-remote https://github.com/sqlite/sqlite.git "refs/tags/$SQLITE_REF" | head -n 1 | cut -f 1) | ||
| fi | ||
|
|
||
| if [ -z "$SQLITE_SHA" ]; then | ||
| # Fallback: assume it's a SHA if ls-remote didn't find it as a ref | ||
| SQLITE_SHA="$SQLITE_REF" | ||
| fi | ||
|
|
||
| echo "sqlite_ref=$SQLITE_REF" >> $GITHUB_OUTPUT | ||
| echo "sqlite_sha=$SQLITE_SHA" >> $GITHUB_OUTPUT | ||
| echo "branch_name=update-sqlite-wasm-${SQLITE_SHA}" >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Check if branch exists | ||
| id: check-branch | ||
| run: | | ||
| BRANCH_NAME="${{ steps.resolve-ref.outputs.branch_name }}" | ||
| if git ls-remote --exit-code --heads origin "$BRANCH_NAME"; then | ||
| echo "Branch $BRANCH_NAME already exists. Skipping build." | ||
| echo "skip=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "skip=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| - name: Set up Buildx Docker CLI plugin | ||
| if: steps.check-branch.outputs.skip != 'true' | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Build Docker image | ||
| if: steps.check-branch.outputs.skip != 'true' | ||
| uses: docker/build-push-action@v6 | ||
| with: | ||
| context: . | ||
| load: true | ||
| tags: sqlite-wasm-builder:env | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max | ||
|
|
||
| - name: Run build | ||
| if: steps.check-branch.outputs.skip != 'true' | ||
| run: | | ||
| mkdir -p out src/bin | ||
| docker run --rm \ | ||
| -e SQLITE_REF="${{ steps.resolve-ref.outputs.sqlite_ref }}" \ | ||
| -e HOST_UID="$(id -u)" \ | ||
| -e HOST_GID="$(id -g)" \ | ||
| -v "$(pwd)/out":/out \ | ||
| -v "$(pwd)/src/bin":/src/bin \ | ||
| sqlite-wasm-builder:env build | ||
| # Fallback fix for permissions if chown inside docker failed or was incomplete | ||
| sudo chown -R $(id -u):$(id -g) out src/bin | ||
|
|
||
| - name: Check for changes | ||
| if: steps.check-branch.outputs.skip != 'true' | ||
| id: git-check | ||
| run: | | ||
| # Add files that might be newly created or modified | ||
| git add src/bin | ||
| git status --porcelain | ||
| if [ -n "$(git status --porcelain)" ]; then | ||
| echo "changes=true" >> $GITHUB_OUTPUT | ||
| else | ||
| echo "changes=false" >> $GITHUB_OUTPUT | ||
| fi | ||
|
|
||
| - name: Create Pull Request | ||
| if: | ||
| steps.check-branch.outputs.skip != 'true' && | ||
| steps.git-check.outputs.changes == 'true' | ||
| uses: peter-evans/create-pull-request@v6 | ||
| with: | ||
| token: ${{ secrets.GITHUB_TOKEN }} | ||
| commit-message: | ||
| 'chore: update SQLite Wasm binaries from ${{ | ||
| steps.resolve-ref.outputs.sqlite_ref }} (${{ | ||
| steps.resolve-ref.outputs.sqlite_sha }})' | ||
| title: | ||
| 'chore: update SQLite Wasm binaries from ${{ | ||
| steps.resolve-ref.outputs.sqlite_ref }}' | ||
| body: | | ||
| This PR updates the SQLite Wasm binaries in `src/bin` by building them from SQLite reference `${{ steps.resolve-ref.outputs.sqlite_ref }}` (commit `${{ steps.resolve-ref.outputs.sqlite_sha }}`). | ||
|
|
||
| Triggered by manual workflow dispatch. | ||
| branch: ${{ steps.resolve-ref.outputs.branch_name }} | ||
| base: main | ||
| delete-branch: true |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,112 @@ | ||
| name: CI | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: read | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| pull_request: | ||
| branches: [main] | ||
|
|
||
| concurrency: | ||
| group: ${{ github.workflow }}-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| defaults: | ||
| run: | ||
| shell: bash | ||
|
|
||
| jobs: | ||
| type-check: | ||
| name: Type check | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repo | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Set up Node.js | ||
| uses: ./.github/actions/node-setup | ||
|
|
||
| - name: Run type check | ||
| run: npm run check-types | ||
|
|
||
| format: | ||
| name: Format check | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repo | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Setup environment (node + install) | ||
| uses: ./.github/actions/node-setup | ||
|
|
||
| - name: Run format check | ||
| run: npx prettier . --check | ||
|
|
||
| test-browser: | ||
| name: Run browser tests | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - name: Checkout repo | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Setup environment (node + install) | ||
| uses: ./.github/actions/node-setup | ||
|
|
||
| - name: Get Playwright version | ||
| id: playwright-version | ||
| run: | ||
| echo "PLAYWRIGHT_VERSION=$(node -e | ||
| "console.log(require('./package-lock.json').packages['node_modules/playwright'].version)")" | ||
| >> $GITHUB_OUTPUT | ||
|
|
||
| - name: Cache Playwright browsers | ||
| uses: actions/cache@v4 | ||
| id: playwright-cache | ||
| with: | ||
| path: ~/.cache/ms-playwright | ||
| key: | ||
| ${{ runner.os }}-playwright-${{ | ||
| steps.playwright-version.outputs.PLAYWRIGHT_VERSION }} | ||
| restore-keys: | | ||
| ${{ runner.os }}-playwright- | ||
|
|
||
| - name: Install Playwright browsers | ||
| if: steps.playwright-cache.outputs.cache-hit != 'true' | ||
| run: npx playwright install --with-deps | ||
|
|
||
| - name: Install Playwright dependencies | ||
| if: steps.playwright-cache.outputs.cache-hit == 'true' | ||
| run: npx playwright install-deps | ||
|
|
||
| - name: Run browser tests | ||
| run: npm run test:browser | ||
|
|
||
| test-node: | ||
| name: Run node tests (Node ${{ matrix.node-version }}) | ||
| runs-on: ubuntu-latest | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| node-version: [20, 22, 23, 24, 25] | ||
| steps: | ||
| - name: Checkout repo | ||
| uses: actions/checkout@v6 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Setup environment (node + install) | ||
| uses: ./.github/actions/node-setup | ||
| with: | ||
| node-version: ${{ matrix.node-version }} | ||
|
|
||
| - name: Run node tests | ||
| run: npm run test:node | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| /src/bin | ||
| /dist | ||
| /node_modules |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| # syntax=docker/dockerfile:1 | ||
| FROM emscripten/emsdk:latest | ||
|
|
||
| ARG DEBIAN_FRONTEND=noninteractive | ||
|
|
||
| # runtime / build deps | ||
| RUN apt-get update \ | ||
| && apt-get install -y --no-install-recommends \ | ||
| ca-certificates \ | ||
| git \ | ||
| build-essential \ | ||
| autoconf \ | ||
| automake \ | ||
| libtool \ | ||
| m4 \ | ||
| python3 \ | ||
| nodejs \ | ||
| npm \ | ||
| zip \ | ||
| unzip \ | ||
| curl \ | ||
| wabt \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # create useful dirs | ||
| RUN mkdir -p /build /out /src/bin /work | ||
|
|
||
| # copy helper script | ||
| COPY scripts/build-sqlite3-wasm.sh /usr/local/bin/build-sqlite3-wasm.sh | ||
| RUN chmod +x /usr/local/bin/build-sqlite3-wasm.sh | ||
|
|
||
| WORKDIR /work | ||
|
|
||
| ENTRYPOINT ["/usr/local/bin/build-sqlite3-wasm.sh"] | ||
| CMD ["build"] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is 21 missing on purpose?
Assuming it's an oversight, we might be able to automate this to always pick the last
nversions counting down from the latest:Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Plan was to only include non-EOL versions (which I've also messed up, since I included 23, which is EOL). Node 20 is the earliest still-supported version (currently in maintenance mode until ~ May), and if you agree with this approach, of only testing non-EOL versions, we'd then remove it in May, along with Node 25 a month later.
Would this be okay?