fix(scripts): honor SPECKIT_PYTHON override for preset manifest parsing

[chelsealong]

Summary

Fixes #4443.

Preset composition raises PyYAML is required to resolve preset template composition whenever the resolve/setup scripts are invoked under a bare
python3 that lacks PyYAML, while a uv tool install / pipx-isolated
CLI venv (which does have PyYAML, per pyproject.toml) is invisible to
that bare interpreter. This happens both when the bash/PowerShell twins
shell out to Python to parse a preset.yml manifest, and when the Python
twin (resolve_template.py / setup_plan.py / etc., via common.py)
tries to import yaml directly in its own process.

SPECKIT_PYTHON is already the established override for exactly this class
of problem elsewhere in the codebase (see
extensions/agent-context/scripts/bash/update-agent-context.sh and its
tests). This PR extends that same override to the three
template-resolution twins:

• scripts/bash/common.sh: _python3_command() now prefers SPECKIT_PYTHON
  (verified to be a working Python 3 interpreter that also has PyYAML)
  before falling back to python3 / python / py -3 on PATH.
• scripts/powershell/common.ps1: Get-Python3Command does the same.
• scripts/python/common.py: since this twin runs in-process rather than
  shelling out, _preset_template_layer now tries import yaml first and,
  if that fails, delegates manifest parsing to SPECKIT_PYTHON via a small
  subprocess-based safe_load/YAMLError shim (_import_yaml /
  _DelegatedYAML) so the rest of the function's logic is unchanged.

No dependencies, lockfiles, or CI config were touched — pyyaml was already
a declared dependency; this only fixes which interpreter picks it up.

Update: an earlier revision of this PR let the bash/PowerShell twins
adopt SPECKIT_PYTHON as soon as it resolved to any Python 3 interpreter,
without checking it actually had PyYAML — unlike the precedent this PR
cites, which validates both. That meant a SPECKIT_PYTHON set to a valid
but PyYAML-less interpreter (e.g. pinned for an unrelated reason) would
shadow a working python3 on PATH and turn previously-succeeding preset
composition into a PyYAML is required failure. Both _python3_command()
and Get-Python3Command now also probe import yaml before accepting
SPECKIT_PYTHON, matching update-agent-context.sh's candidate-validation
loop; if the probe fails they fall through to the existing python3/
python/py -3 chain exactly as if SPECKIT_PYTHON were unset.

Test plan

Added test_all_variants_honor_speckit_python_override_when_yaml_missing to
tests/test_resolve_template_python_parity.py. It creates a real
PyYAML-less interpreter (python -m venv --without-pip) and:

1. runs the bash and Python twins with that interpreter as the default
   python3 and no SPECKIT_PYTHON set — asserts both fail (reproduces
   [Bug] Preset composition fails under isolated installs: scripts run on system python3 but PyYAML lives in the CLI venv #4443), then
2. sets SPECKIT_PYTHON to the interpreter that has PyYAML — asserts both
   succeed and produce the correct composed template content.
   (PowerShell is included in the same assertions when pwsh is available.)

Also added test_all_variants_fall_back_when_speckit_python_lacks_pyyaml,
covering the regression an earlier revision introduced: it sets
SPECKIT_PYTHON to a real Python-3 interpreter without PyYAML while
leaving PATH's own python3 (which has PyYAML) untouched, and asserts
composition still succeeds via that PATH fallback across the bash and
PowerShell twins.

Confirmed both tests fail without their respective fixes:

• test_all_variants_honor_speckit_python_override_when_yaml_missing: with
  git checkout HEAD~2 -- scripts/bash/common.sh scripts/powershell/common.ps1 scripts/python/common.py, fails with
  assert False on the override-succeeds assertion.
• test_all_variants_fall_back_when_speckit_python_lacks_pyyaml: with only
  the latest commit's two files reverted (git checkout HEAD~1 -- scripts/bash/common.sh scripts/powershell/common.ps1, i.e. the state the
  independent reviewer blocked), fails the same way — reproducing the
  reviewer's exact regression scenario.
  Restored the fix in both cases and reran; both pass.

$ .venv/bin/python -m pytest tests/test_resolve_template_python_parity.py -q
============================= 42 passed in 39.14s ==============================

$ .venv/bin/python -m pytest tests/extensions/test_extension_agent_context.py -q
======================== 39 passed, 1 skipped in 10.62s ========================

$ .venv/bin/python -m pytest tests/ -q
========== 7706 passed, 12 skipped, 48 warnings in 416.00s (0:06:56) ===========

$ shellcheck --severity=error scripts/bash/common.sh
(no output, exit 0)

$ uvx ruff@0.15.0 check tests/test_resolve_template_python_parity.py src
All checks passed!

AI disclosure

This PR was written by an autonomous coding agent (Claude/Anthropic) based on
the issue's description and existing SPECKIT_PYTHON precedent in the
codebase. An independent reviewer agent blocked the first revision for
letting the bash/PowerShell twins honor SPECKIT_PYTHON without checking
it actually had PyYAML (a regression relative to the cited precedent); this
revision fixes that specific gap, adds a regression test reproducing it,
and reruns the full test suite, shellcheck, and ruff before pushing.

[Copilot]

🟡 Changes recommended

Multiple moderate portability and manifest-parsing issues remain unresolved.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Extends SPECKIT_PYTHON support to preset manifest parsing across all script variants.

Changes:

• Selects PyYAML-capable override interpreters in Bash and PowerShell.
• Delegates YAML parsing from Python when needed.
• Adds override and fallback parity tests.

File summaries

File	Review
tests/test_resolve_template_python_parity.py	Moderate: both tests use POSIX-only venv paths; baseline also fails to unset SPECKIT_PYTHON.
scripts/python/common.py	Moderate: JSON serialization rejects valid YAML values such as timestamps.
scripts/powershell/common.ps1	Override selection reviewed without additional findings.
scripts/bash/common.sh	Moderate: caller argument splitting breaks interpreter paths containing spaces.

Review details

• Files reviewed: 4/4 changed files
• Comments generated: 5
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[chelsealong]

Addressed all 5 findings from the Copilot review:

• scripts/bash/common.sh: _python3_command now emits one argv element per line (mapfile -t), so a SPECKIT_PYTHON path containing spaces is no longer split across read -r -a.
• scripts/python/common.py: the delegated-YAML subprocess now dumps with default=str, so non-JSON-native YAML values (e.g. an unquoted date) no longer crash the parse.
• tests/test_resolve_template_python_parity.py: the two SPECKIT_PYTHON tests now pick venv paths via a new venv_python3_exe() helper (POSIX bin/python3 vs Windows Scripts/python.exe), and explicitly unset SPECKIT_PYTHON before building the baseline env. Added two new regression tests covering the spaces-in-path and non-JSON-YAML-value fixes.

Verified both new tests fail on the pre-fix code (git checkout HEAD~1 -- <file>) and pass after; full suite (pytest tests/ -q) is green at 7708 passed.

[Copilot]

🟡 Changes recommended

Bash 3.2 incompatibility and delegated YAML transport/validation regressions remain.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

scripts/python/common.py:412

• text=True uses the process locale for the pipe. Under the repository's tested ASCII-locale configuration, a valid manifest containing non-ASCII metadata raises UnicodeEncodeError before the delegated interpreter can parse it. Force UTF-8 for both the parent pipe and the child Python stdio.

scripts/bash/common.sh:520

• This second mapfile call also breaks under the supported macOS system Bash 3.2, so preset lookup cannot use the selected interpreter there. Use the Bash-3-compatible while read array population while retaining each emitted command argument as a single element.

        mapfile -t python_cmd < <(_python3_command)

scripts/bash/common.sh:640

• mapfile requires Bash 4+, but sh is the default on macOS where /bin/bash is 3.2. Consequently composed preset resolution still fails on the environment reported by #4443 even when SPECKIT_PYTHON is valid; use the portable while read form here as well.

        mapfile -t python_cmd < <(_python3_command)

• Files reviewed: 5/5 changed files
• Comments generated: 2
• Review effort level: Balanced