Skip to content

Latest commit

 

History

History
428 lines (309 loc) · 10 KB

DEVELOPMENT.md

File metadata and controls

428 lines (309 loc) · 10 KB

wsianon Development Notes

This file is used to track the initial experimentation with the build tools.

Building Python Package

New module code written in src/wsianon directory.

Syncing C library from remote

# TODO: @luciorq `macos-support` has already been merged in
# + the main (master) branch.
# Initial clone: `git clone -b macos-support https://github.com/luciorq/wsi-anon ./temp/`;

git clone -b macos-support https://github.com/luciorq/wsi-anon ./temp/

rm -rf ./include/*;
cp -r temp/src/* ./include/;
cp temp/LICENSE ./include/;
rm -rf ./temp;

Test data

Test data is acquired from the Openslide project.

Adding features to Upstream project on GitLab

Adding suport for MacOS in the Upstream EMPAIA WSI Anon project on GitLab.

Fork to https://gitlab.com/luciorq/wsi-anon

git clone https://gitlab.com/luciorq/wsi-anon ~/projects/forks/wsi-anon;
cd ~/projects/forks/wsi-anon;
git branch;
git checkout -b <BRANCH_NAME>;

Make necessary edits to the branch.

Check if everything is working with make.

make

Commit and push.

git add -p <FILES_TO_ADD>
git commit -m "...";

git push origin <BRANCH_NAME>

This will push the changes to your personal fork, you still need to create the Pull Request in the original repository.

Testing WSI Anon Python Wrapper

conda create -n wsi-anon-env -c conda-forge -c bioconda openslide python pytest tiffslide openslide-python


cp bin/libwsianon.dylib ./
conda run -n wsi-anon-env python -m pytest ./wrapper/python/test

Add additional remote on GitHub

git remote -v;
git remote add origin-gh https://github.com/luciorq/wsi-anon;
git remote -v;
git push origin-gh --all;

gh repo view --web;

Install docs and test tools

Trying pixi with pip for installing deps;

pixi add python pip;
pixi run python -m pip install -e ".[examples,docs,tests]";
pixi run python -m pytest -vvv;

After accepting Pull Request

git remote prune --dry-run origin
git remote prune origin
alias git-list-untracked='git fetch --prune && git branch -r | awk "{print \$1}" | grep -E -v -f /dev/fd/0 <(git branch -vv | grep origin) | awk "{print \$1}"'
git-list-untracked
alias git-remove-untracked='git fetch --prune && git branch -r | awk "{print \$1}" | grep -E -v -f /dev/fd/0 <(git branch -vv | grep origin) | awk "{print \$1}" | xargs git branch -d'
git-remove-untracked

Fix tests using openslide and tiffslide

pixi add ruff;
pixi run python -m pip install -e ".[examples,docs,tests]";
pixi run ruff check;
pixi run python -m pytest -vvv;

Updating After Merge and Update of Upstream

Working again on this project for the URSSI 2025 Summer School.

Date: 2025-08-18.

# cd ~/projects/wsianon

git -C ~/projects/forks/wsi-anon/ status
git -C ~/projects/forks/wsi-anon/ remote -v
# origin  https://gitlab.com/luciorq/wsi-anon
# origin-gh       https://github.com/luciorq/wsi-anon
# upstream        https://gitlab.com/empaia/integration/wsi-anon

# Go to <https://gitlab.com/luciorq/wsi-anon> and make sure to sync fork
git -C ~/projects/forks/wsi-anon/ fetch
git -C ~/projects/forks/wsi-anon/ branch

git -C ~/projects/forks/wsi-anon/ pull

git -C ~/projects/forks/wsi-anon/ log \
  --graph \
  --pretty=format:"%C(magenta)%h%Creset -%C(red)%d%Creset %s %C(dim green)(%cr) %C(cyan)<%an>%Creset" \
  --abbrev-commit;

# Prepare build project
ls -lah ./include;
rm -rf ./include/*;
ls -lah ~/projects/forks/wsi-anon/src/;
cp -r ~/projects/forks/wsi-anon/src/* ./include/;
ls -lah ~/projects/forks/wsi-anon/;
cp ~/projects/forks/wsi-anon/LICENSE ./include/;
rm -rf ./temp;
# pixi run python -m pip install -e ".[examples,docs,tests]";
# pixi run ruff check;
# pixi run python -m pytest -vvv;
pixi init;
pixi add python uv;
pixi run uv pip install -e ".[cli]";

Using uv

# uv pip install --extra cli .;
pixi run uv pip install -e ".[cli]";

pixi run uv tool run ruff check;
pixi run uv tool run rumdl check;
pixi run uv run --group docs mkdocs;
pixi run uv run --group tests pytest;

Using Pixi only with embedded uv

Date: 2025-08-25.

Testing 100% pixi managed development, but using PyPI dependencies.

Remove uv and pip from env.

pixi remove pip
pixi remove uv

Or starting fresh with pixi on a project that already have pyproject.toml.

# maybe add `-c ome` because of `bftools` (bioformats)
pixi init --scm github -p "win-64" -p "linux-64" -p "osx-arm64" -p "osx-64" --format pyproject -c conda-forge -vv

Example output.

|> pixi init --scm github -p "win-64" -p "linux-64" -p "osx-arm64" -p "osx-64" --format pyproject -c conda-forge -vv
DEBUG pixi_config: Loading config from /etc/pixi/config.toml
DEBUG pixi_config: Loading config from /Users/luciorq/.config/pixi/config.toml
DEBUG pixi_config: Loading config from /Users/luciorq/Library/Application Support/pixi/config.toml
DEBUG pixi_config: Loading config from /Users/luciorq/.pixi/config.toml
DEBUG pixi_config: Loaded config from: /Users/luciorq/.pixi/config.toml
✔ Added package 'wsianon' as an editable dependency.
✔ Added environments 'examples', 'docs', 'cli', 'tests' from optional dependencies or dependency groups.

Snippet from pyproject.toml.

[...]

[tool.pixi.workspace]
channels = ["conda-forge"]
platforms = ["win-64", "linux-64", "osx-arm64", "osx-64"]

[tool.pixi.pypi-dependencies]
wsianon = { path = ".", editable = true }

[tool.pixi.environments]
default = { solve-group = "default" }
cli = { features = ["cli"], solve-group = "default" }
docs = { features = ["docs"], solve-group = "default" }
examples = { features = ["examples"], solve-group = "default" }
tests = { features = ["tests"], solve-group = "default" }

[tool.pixi.tasks]

The following commands should be transformed in tasks.

# TODO: Add actual tasks to run linting, docs, and tests.

# Task that simulate:
pixi run -e lint ruff check

pixi run -e lint rumdl check

# Run Tests
pixi run -e tests python -m pytest -vvv

# Run Docs
pixi run -e docs python -m mkdocs build

# Run CLI
pixi run wsianon --help

pixi run -e cli wsianon --help

pixi run python -m wsianon --help
pixi run python -m wsianon

Adding tasks to run on each environment.

pixi tasks ...

Removing setup.py

Moving setup.py Extension table to pyproject.toml.

[build-system]
requires = [
  "setuptools>=80",
  "setuptools-scm>=8"
]
build-backend = "setuptools.build_meta"

[tool.setuptools]
ext-modules = [
{ name = "wsianon.libwsianon", sources = [
  "include/aperio-flavor-io.c",
  "include/b64.c",
  "include/buf.c",
  "include/conf.c",
  # "include/console-app.c",
  "include/enc.c",
  "include/hamamatsu-io.c",
  "include/huff.c",
  "include/ini-parser.c",
  "include/isyntax-io.c",
  # "include/js-file.c",
  "include/mirax-io.c",
  "include/native-file.c",
  "include/philips-based-io.c",
  "include/philips-tiff-io.c",
  "include/tiff-based-io.c",
  "include/utils.c",
  "include/ventana-io.c",
  # "include/wsi-anonymizer-wasm.c",
  "include/wsi-anonymizer.c"
], language = "c" }
]

Until now I couldn't find the the right of using globs or patterns in the sources list.