conda-pack 0.9.0+ silently corrupts Python source files containing `\\?\` on Windows

[AidanShipperley]

Checklist

• I added a descriptive title
• I searched open reports and couldn't find a duplicate

What happened?

PR #432 (released in 0.9.0) fixed #398, where Windows extended-length path prefixes (\\?\) were leaking into packed files like conda-hook.ps1. Two of the three changes are well-scoped:

1. core.py L1201–L1205 (and L1223–L1227): Normalizes \\?\ / //?/ out of prefix placeholders before replacement.
2. prefixes.py update_prefix() L68–L69: Strips //?/ from the new prefix for .ps1 files.

However, a third change in prefixes.py text_replace() unconditionally strips the raw byte sequences \\?\ and //?/ from all text file data on Windows:

conda-pack/conda_pack/prefixes.py

Lines 114 to 118 in 7931570

	if on_win:
	# Replace \\?\ with empty string (remove extended-length prefix)
	data = data.replace(b'\\\\?\\', b'')
	# Replace //?/ with empty string (Windows extended-length prefix with forward slashes)
	data = data.replace(b'//?/', b'')

This does not distinguish between conda prefix paths and legitimate source code that references \\?\. Any .py file containing those bytes is silently corrupted during packing.

Example

huggingface_hub/file_download.py contains:

and not os.path.abspath(lock_path).startswith("\\\\?\\")

The raw bytes of "\\\\?\\" in the .py file are \ \ \ \ ? \ \ (7 bytes). The 4-byte pattern \ \ ? \ (b'\\\\?\\') matches at offset 2. After removal, three backslashes remain and the source becomes:

and not os.path.abspath(lock_path).startswith("\\\")

Python parses \\ as an escaped backslash, then \" as an escaped quote, and the string never terminates:

  File "...\huggingface_hub\file_download.py", line 1xxx
    and not os.path.abspath(lock_path).startswith("\\\")
                                                  ^
SyntaxError: unterminated string literal

Any package referencing \\?\ in source code is affected. As more packages add Windows long-path support, this will silently break more environments over time. This is especially damaging for offline/air-gapped deployments where the packed environment cannot be repaired after transfer.

Root cause

The blanket strip exists to clean up dangling prefixes left after the core.py normalization. When core.py normalizes a placeholder from //?/C:\envs\myenv to C:\envs\myenv, the standard replacement matches the path within an extended-prefix occurrence in the file data, leaving the prefix dangling:

File data:       //?/C:\envs\myenv\lib\...
Placeholder:     C:\envs\myenv              ← normalized by core.py
New prefix:      C:\dest\prefix

After replace:   //?/C:\dest\prefix\lib\...
                 ^^^^
                 dangling prefix

The blanket strip cleans this up, but it's too broad; it matches \\?\ in all contexts, not just adjacent to conda prefixes.

Suggested fix

Replace the blanket strip with targeted replacement that only matches extended-length prefixes immediately followed by the actual placeholder:

def text_replace(data, placeholder, new_prefix):
    placeholder_bytes = placeholder.encode('utf-8')
    new_prefix_bytes = new_prefix.encode('utf-8')

    if on_win:
        # Replace extended-prefix + placeholder as a unit FIRST,
        # so the prefix doesn't dangle after the standard replacement.
        data = data.replace(b'\\\\?\\' + placeholder_bytes, new_prefix_bytes)
        data = data.replace(b'//?/' + placeholder_bytes, new_prefix_bytes)

    # Standard replacement
    data = data.replace(placeholder_bytes, new_prefix_bytes)

    return data

This handles all three cases without touching unrelated source code:

• \\?\C:\envs\myenv → replaced cleanly
• //?/C:\envs\myenv → replaced cleanly
• C:\envs\myenv → standard replacement

The existing test test_windows_extended_length_path_cleanup would need to be updated to reflect the targeted behavior (it currently asserts a blanket strip).

Reproducing

1. Create a conda environment on Windows containing huggingface_hub>=0.24 (or any package referencing \\?\ in source)
2. Pack with conda-pack>=0.9.0 using --dest-prefix
3. Unpack on another machine (or same machine at a different path)
4. import huggingface_hub → SyntaxError

Workaround: Pin conda-pack==0.8.1.

Additional Context

• conda-pack version: 0.9.0 and 0.9.1
• Platform: Windows Server 2019 / Windows 10+
• Python: 3.12.9
• Original issue fixed by PR Fix issue with extended path format on Windows #432: Env with conda dist included results in conda-hook.ps1 content with extended path format on Windows? #398
• PR that introduced the regression: Fix issue with extended path format on Windows #432