-
Notifications
You must be signed in to change notification settings - Fork 32
Open
Description
Description
When using S3 channels (e.g., s3://bucket-name/linux-64), package URLs are corrupted to .filename instead of the full S3 URL.
Root Cause
In shards.py, the base_url property uses Python's urllib.parse.urljoin:
@property
def base_url(self) -> str:
return urljoin(urljoin(self.url, self._base_url), ".")However, urljoin doesn't recognize s3:// as a valid URL scheme:
>>> from urllib.parse import urljoin
>>> urljoin('https://repo.anaconda.com/pkgs/main/linux-64', '.')
'https://repo.anaconda.com/pkgs/main/' # Correct!
>>> urljoin('s3://minio-s3-server/linux-64', '.')
'.' # WRONG - should be 's3://minio-s3-server/'Then in index.py:621, the URL is constructed via string concatenation:
url=f"{base_url}{filename}"When base_url is ., this produces .zlib-1.2.11-h7b6447c_3.tar.bz2 instead of s3://bucket/linux-64/zlib-1.2.11-h7b6447c_3.tar.bz2.
Error Message
conda.CondaMultiError: No connection adapters were found for '.zlib-1.2.11-h7b6447c_3.tar.bz2'
Steps to Reproduce
- Set up a local S3 server (e.g., Minio)
- Create a conda channel on S3
- Try to install a package from the S3 channel with libmamba solver
conda create -n test --channel s3://bucket-name zlibWorkaround
Use the classic solver:
CONDA_SOLVER=classic conda create -n test --channel s3://bucket-name zlibSuggested Fix
The base_url property needs to handle URL schemes that urljoin doesn't recognize (like s3://, ftp://, etc.). One approach:
from urllib.parse import urlparse, urljoin
@property
def base_url(self) -> str:
parsed = urlparse(self.url)
# urljoin doesn't handle non-http schemes properly
if parsed.scheme and parsed.scheme not in ('http', 'https', ''):
# For non-http schemes, manually construct the base URL
if self._base_url:
return urljoin(self.url, self._base_url)
# Ensure trailing slash for proper concatenation
return self.url.rstrip('/') + '/'
return urljoin(urljoin(self.url, self._base_url), ".")Environment
- conda version: 26.1.0
- conda-libmamba-solver version: (current)
- Python: 3.13
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels