Editable pip installs for devs

[Helveg]

Context

I'd like to have NEURON pip installed coupled to my git repository using editable installs: pip install -e ..

Overview of the issue

When I run pip install -e . the following error is thrown while I have Cython and numpy:

(nrn-dev) robin@LAPTOP-N6QKV63M:~/git/nrn$ pip install -e .
Obtaining file:///mnt/c/Users/robin/Documents/GIT/nrn
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/bin/python3.9 /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpn5uj8iah
       cwd: /mnt/c/Users/robin/Documents/GIT/nrn
  Complete output (1 lines):
  ERROR: RX3D wheel requires Cython and numpy. Please install beforehand
  ----------------------------------------
WARNING: Discarding file:///mnt/c/Users/robin/Documents/GIT/nrn. Command errored out with exit status 1: /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/bin/python3.9 /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpn5uj8iah Check the logs for full command output.
ERROR: Command errored out with exit status 1: /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/bin/python3.9 /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpn5uj8iah Check the logs for full command output.
(nrn-dev) robin@LAPTOP-N6QKV63M:~/git/nrn$ pip show cython numpy
Name: Cython
Version: 0.29.22
Summary: The Cython compiler for writing C extensions for the Python language.
Home-page: http://cython.org/
Author: Robert Bradshaw, Stefan Behnel, Dag Seljebotn, Greg Ewing, et al.
Author-email: cython-devel@python.org
License: Apache
Location: /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/lib/python3.9/site-packages
Requires:
Required-by:
---
Name: numpy
Version: 1.20.1
Summary: NumPy is the fundamental package for array computing with Python.
Home-page: https://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /home/robin/.pyenv/versions/3.9.0/envs/nrn-dev/lib/python3.9/site-packages
Requires:
Required-by: NEURON

FWIW pip install . has the same issue.

Expected result/behavior

An editable install

NEURON setup

• Version: master branch
• Installation method pip install -e .
• OS + Version: WSL2 + Ubuntu 20.04
• Compiler + Version: gcc-9

[alexsavulescu]

Not straightforward given the complexity of the package (C/C++ extensions, RXD + Cython, ...).

[Helveg]

Editable installs are now supported by https://www.python.org/dev/peps/pep-0660/, so PEP517 compliant builds can be ran in editable mode (cibuildwheel uses PEP517 as well, so if it works there it should work editable). If you declare your entire build in accordance with PEP517 (which one can hope is as simple as declaring cython and numpy as build dependencies) could yield functional editable installs.

[JCGoran]

It's been a long time since the last reply to this discussion, so I would like to chime in a bit with recent progress. Since merging #3350, NEURON switched to the PEP 517-compliant scikit-build-core build backend, which basically acts as a thin wrapper around CMake. Consequently, the following should now more or less work as an editable install:

pip install -e . --no-build-isolation --config-settings=build-dir=[dir]

All of the build files will then be placed in [dir], and the NEURON Python package will be importable in the currently active virtual environment (on MacOS, due to certain CMake issues with resolving paths, the value of build-dir is by default build_wheel, while on Linux one needs to set it manually since the default is to use a temp dir). Editing the sources and running the above again drastically reduces build times compared to building the wheel from scratch. To speed things up even further, one could in principle use ccache by passing an additional --config-settings=cmake.define.CMAKE_C_COMPILER_LAUNCHER=ccache --config-settings=cmake.define.CMAKE_CXX_COMPILER_LAUNCHER=ccache to the above, though I haven't tested it.

[Helveg]

That's great news! Why is --no-build-isolation required?

[nrnhines]

That's great news! Why is --no-build-isolation required?

I wondered the same thing. https://grok.com/share/bGVnYWN5_bcbec2c2-02ab-4dda-b965-557bef373cf2 appears reasonable though verbose. Verifying it is not BS, though, is further than I want to go at the moment.

[JCGoran]

For --no-build-isolation, see: https://scikit-build-core.readthedocs.io/en/latest/configuration/index.html#editable-installs