rez-bind python & rez-release depends-on-python: should python package be released?

[alexey-pelykh]

It's more of the what-is-by-design question. The situation is as follows:

1. by default, rez binds python (and os, arch, platform) package to local_packages_path
2. when a package that depends on python is released, the process assumes that all dependencies are released as well, thus forcing to release locally-bound python, os, arch, platform
3. rez-pip by default looks for the latest python & pip packages and corresponding versions may not be present locally

Seems like a collision.

Workaround attempt no. 1: Specify the exact version for rez-pip that matches locally-bound python

Won't help since the same version can be released from different arch, producing

The following package conflicts occurred: (arch-AMD64 <--!--> arch-x86_64)

Workaround attempt no. 2: Remove all published python package versions, forcing it to use locally-bound

• would help for rez-pip
• would help with rez env resolving unless --no-local is specified
• will cause issue with rez-release since not all dependencies are published

Questions:

• by design, what should be the intended approach?
• rez-bind is going to be deprecated (so I've heard)?
• should all possible combinations of python-os-arch-platform be published to shared (cross-platform) rez packages repo?

Thanks in advance for the guidance!

cc @nerdvegas

[instinct-vfx]

I am not sure i really understand the problem. Trying to unravel a little.
First of all, yes rez-bind is going to be deprecated as pretty much everything it does can be done with "normal" packages these days.

Secondly, you don't have to locally bind python necessarily. It can be a central package only (it might still wrap a local installation which is sometimes referred to as a "native" package, or it might carry the whole release as payload). There is no real difference between local_packages_path and packages_path except for local_packages_path being the default target for installation (this is also consistent for rez-build and rez-pip, this is not rez-pip specific). The reason is that a build by default is considered local and dev and only using rez-release or rez-pip -r targets your release_packages_path. To re-iterate: Using rez-pip without -r or using rez-build or using rez-bind without -r is intended to release local dev packages. Not production use packages.

Also note that os, arch and platform are a somewhat special case as they are by default configured as implicit packages, so they are implicitly added to every resolve.

rez-pip does not just look for "the latest python and pip packages". It has the following search order (from the docstring):

 1. Search for rezified python matching python version request;
        2. If found, test if pip is present;
        3. If pip is present, use it;
        4. If not present, search for rezified pip (this is for backwards compatibility);
        5. If rezified pip is found, use it;
        6. If not, fall back to rez's python installation.

It does not matter if the desired versions do not exist locally if they exist centrally. They do have to exist "somewhere" of course for this to work.

I think the main misunderstanding here resolves around local_packages_path vs. packages_path. Always treat local_packages_path as dev and basically only for testing and release everything you intent to really use to packages_path. (Not it does not necessarily matter if either of these are local or central. Both is possible technically but i would strongly recommend to use local_packages_path locally and only while developing or testing and make packages_path a central shared repository with production packages that can also be used by everyone else.

You will have to release all combinations of os/arch/platform to a shared location. And this is also true for python (but as it is not an implicit by default that is no different from any other package out there, say a DCC).

If you are starting out i would also suggest to skip rez-bind and not build new packages to be used with bind. If you are on windows, i would also advise not to use a rez-bind created python package as that comes with a bunch of restrictions and pitfalls on windows. For windows specifically i did a live exercise recently with a bunch of people to walk them through creating a tool to create full python rez packages on windows with full payload and turn that into an automatic tool. There sadly is no recording available but i did share the tool. I might follow up with a tutorial, but i can not promise that. You can find the tool here: https://github.com/techartorg/rez_utils/tree/main/rezify_python

[instinct-vfx]

P.S. i removed the bug label because i don't think this can be considered a bug no matter how you look at it. Personally i also think this would be better suited for Github Discussions rather than an issue.