Skip to content

Don't install pypi packages to ~/.local #1337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
danielhollas wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Don't install pypi packages to ~/.local #1337

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
6f6677d
Don't install pypi packages to ~/.local
danielhollas Jun 5, 2025
034434b
Exclude '.cache' directory from home.tar
danielhollas Jun 5, 2025
65825bc
Point to origin
danielhollas Jun 5, 2025
a69d7d0
Install vibroscopy with uv
danielhollas Jul 2, 2025
6967356
Unpin spglib
danielhollas Jul 18, 2025
f98b13d
pip install aiida-bader
danielhollas Nov 20, 2025
7161641
Install vibroscopy via pip
danielhollas Nov 21, 2025
0cf153d
pip install vibroscopy
danielhollas Nov 21, 2025
b6e560d
Try this?
danielhollas Nov 21, 2025
88b9bd8
Test python 3.12 and aiida-core=2.8.0 (#1466)
danielhollas Mar 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@ jobs:

strategy:
matrix:
python-version: ['3.9', '3.11']
aiida-core-version: ['2.7']
python-version: ['3.9', '3.12']
aiida-core-version: ['2.8']
fail-fast: false

runs-on: ubuntu-latest
Expand All @@ -36,15 +36,15 @@ jobs:
steps:

- name: Check out app
uses: actions/checkout@v4
uses: actions/checkout@v6

- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}

- name: Install uv
uses: astral-sh/setup-uv@v6
uses: astral-sh/setup-uv@v7
with:
activate-environment: true
python-version: ${{ matrix.python-version }}
Expand All @@ -56,7 +56,7 @@ jobs:
run: pytest -v tests --cov=aiidalab_qe --skip-slow

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4
uses: codecov/codecov-action@v5
with:
flags: python-${{ matrix.python-version }}
token: ${{ secrets.CODECOV_TOKEN }}
42 changes: 23 additions & 19 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# 1) Global ARGs
###############################################################################
ARG FULL_STACK_VER=2025.1027
ARG UV_VER=0.8.4
ARG UV_VER=0.11.1
ARG QE_VER=7.4
ARG QE_DIR=/opt/conda/envs/quantum-espresso-${QE_VER}
ARG HQ_VER=0.19.0
Expand All @@ -17,13 +17,10 @@ ARG UV_CACHE_DIR=/tmp/uv_cache
ARG QE_APP_SRC=/tmp/quantum-espresso
ARG COMPUTER_LABEL="localhost"

#
# We'll define the possible HQ download URLs (for x86_64 and ARM64).
#
ARG HQ_URL_AMD64="https://github.com/It4innovations/hyperqueue/releases/download/v${HQ_VER}/hq-v${HQ_VER}-linux-x64.tar.gz"
ARG HQ_URL_ARM64="https://github.com/It4innovations/hyperqueue/releases/download/v${HQ_VER}/hq-v${HQ_VER}-linux-arm64-linux.tar.gz"
ARG VIBROSCOPY_PKG="aiidalab-qe-vibroscopy@git+https://github.com/mikibonacci/aiidalab-qe-vibroscopy@v1.2.0"
ARG MUON_PKG="aiidalab-qe-muon@git+https://github.com/mikibonacci/aiidalab-qe-muon@v1.0.0"
ARG VIBROSCOPY_PKG="aiidalab-qe-vibroscopy@git+https://github.com/aiidalab/aiidalab-qe-vibroscopy@v1.2.1"
Copy link
Copy Markdown
Contributor Author

@danielhollas danielhollas Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@edan-bainglass @mikibonacci please verify that these repositories are in sync.

Copy link
Copy Markdown
Member

@edan-bainglass edan-bainglass Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you mean by in sync. As far as I can see, after moving vibroscopy under the AiiDAlab team, development has only continued there. Correct me if I'm wrong there @mikibonacci.

However, the opposite appears to be true for Muons 🤔

Copy link
Copy Markdown
Contributor Author

@danielhollas danielhollas Mar 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

However, the opposite appears to be true for Muons 🤔

I guess this is what I meant 😉 When I looked originally, it looked like the active development was done on private repo's and not those that are under aiidalab org.

Copy link
Copy Markdown
Member

@edan-bainglass edan-bainglass Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mikibonacci can you clarify? 🙏

ARG MUON_PKG="aiidalab-qe-muon@git+https://github.com/aiidalab/aiidalab-qe-muon@v1.0.1"
ARG AIIDA_HQ_PKG="aiida-hyperqueue~=0.3.0"

###############################################################################
Expand Down Expand Up @@ -112,7 +109,8 @@ ENV UV_CONSTRAINT=${PIP_CONSTRAINT}
# Install the aiida-hyperqueue
RUN --mount=from=uv,source=/uv,target=/bin/uv \
--mount=from=build_deps,source=${UV_CACHE_DIR},target=${UV_CACHE_DIR},rw \
uv pip install --system --strict --cache-dir=${UV_CACHE_DIR} ${AIIDA_HQ_PKG}
uv pip install --system --strict --cache-dir=${UV_CACHE_DIR} \
${AIIDA_HQ_PKG} ${MUON_PKG} aiida-bader

COPY ./before-notebook.d/* /usr/local/bin/before-notebook.d/

Expand All @@ -127,18 +125,16 @@ RUN --mount=from=qe_conda_env,source=${QE_DIR},target=${QE_DIR} \
verdi code create core.code.installed --label python --computer=localhost --default-calc-job-plugin pythonjob.pythonjob --filepath-executable=/opt/conda/bin/python -n && \
verdi code create core.code.installed --label bader --computer=localhost --default-calc-job-plugin bader.bader --filepath-executable=${QE_DIR}/bin/bader -n && \
verdi code create core.code.installed --label wannier90 --computer=localhost --default-calc-job-plugin wannier90.wannier90 --filepath-executable=/opt/conda/bin/wannier90.x -n && \
# Additional plugins
pip uninstall -y phonopy && \
pip install aiida-bader ${VIBROSCOPY_PKG} ${MUON_PKG} && \
pip install --no-user ${VIBROSCOPY_PKG} && \
# run post_install for plugin
python -m aiida_bader post-install && \
python -m aiidalab_qe_vibroscopy setup-phonopy && \
python -m aiidalab_qe_muon setup-python3 && \
# wannier90 plugin need SSSP 1.1
# Install this earlier as part of the pseudo installation
aiida-pseudo install sssp -v 1.1 -x PBE && \
aiida-pseudo install sssp -v 1.1 -x PBEsol && \
verdi daemon stop && \
pip install spglib==2.5.0 && \
mamba run -n aiida-core-services pg_ctl stop && \
touch /home/${NB_USER}/.FLAG_HOME_INITIALIZED && \
# NOTE: The work folder is empty but if included clashes with the work folder in a Renku
Expand All @@ -147,7 +143,7 @@ RUN --mount=from=qe_conda_env,source=${QE_DIR},target=${QE_DIR} \
# It is usually safe (and preferable) to let .conda be recreated on the fly each time,
# because .conda typically just holds local environment information, caches, or references
# to available environments.
cd /home/${NB_USER} && tar -cf /opt/conda/home.tar --exclude work --exclude .conda .
cd /home/${NB_USER} && tar -cf /opt/conda/home.tar --exclude .cache --exclude work --exclude .conda .

###############################################################################
# 6) Final stage
Expand All @@ -166,19 +162,27 @@ ARG MUON_PKG

USER ${NB_USER}
WORKDIR /tmp
# Install python dependencies

# Install common dependencies such as pymatgen and pandas via conda,
# to avoid building them when installing via pip
# TODO: Remove this once it is part of the full-stack image.
RUN mamba install aiida-core.atomic_tools --y && \
mamba clean --all -f -y

# Install dependencies in the final image.
# It is important that these are installed in /opt/conda, not ~/.local
# Use uv cache from the previous build step
# # Install the aiida-hyperqueue
# NOTE: uv refuses the install aiidalab-qe-vibroscopy due to invalid RECORD file,
# pyproject.toml is likely invalid as it contains direct git dependency.
Comment on lines +175 to +176
Copy link
Copy Markdown
Contributor Author

@danielhollas danielhollas Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been fixed

Suggested change
# NOTE: uv refuses the install aiidalab-qe-vibroscopy due to invalid RECORD file,
# pyproject.toml is likely invalid as it contains direct git dependency.

ENV UV_CONSTRAINT=${PIP_CONSTRAINT}
RUN --mount=from=uv,source=/uv,target=/bin/uv \
--mount=from=build_deps,source=${UV_CACHE_DIR},target=${UV_CACHE_DIR},rw \
--mount=from=build_deps,source=${QE_APP_SRC},target=${QE_APP_SRC},rw \
uv pip install --strict --system --compile-bytecode --cache-dir=${UV_CACHE_DIR} \
${QE_APP_SRC} ${AIIDA_HQ_PKG}
# Install plugins in the final image
RUN pip install aiida-bader ${VIBROSCOPY_PKG} ${MUON_PKG} && \
mamba install scipy==1.13.1 --y && \
mamba clean --all -f -y
${QE_APP_SRC} aiida-bader ${AIIDA_HQ_PKG} ${MUON_PKG}

# This fails on arm with `uv pip` so is installed separately
RUN pip install --no-user ${VIBROSCOPY_PKG}

# copy hq binary
COPY --from=home_build /opt/conda/hq /usr/local/bin/
Expand Down
Loading