Extended-card precondition tests can skip a real MUST violation; CARD-EXT-001 retains false-failure class

[millsmillsymills]

Summary

The extended-card precondition tests can silently skip a genuine MUST violation, and CARD-EXT-001 still carries the false-failure class that PR #186 fixed for CARD-EXT-002.

Background

PR #186 re-routed CARD-EXT-002 to a dedicated probe-and-skip test (TestExtendedCardNotConfigured in tests/compatibility/core_operations/test_error_handling.py) because the precondition — a server that declares extendedAgentCard support but has not configured one — is not observable from the agent card. That fix is correct and merged-ready.

Problems

1. Test can pass or skip, never fail. When a server declares support, leaves the extended card unconfigured, but returns the wrong error code (not -32007 / EXTENDED_AGENT_CARD_NOT_CONFIGURED_ERROR), the test pytest.skips instead of failing. A real conformance violation goes unreported. The skip heuristic should be tightened — e.g. only skip on an auth-shaped/precondition-shaped error, and fail on other A2A error codes.

2. CARD-EXT-001 has the same false-failure class. In tck/requirements/agent_card.py, CARD-EXT-001 still carries operation= with no expected_error and expects an AgentCard success response. Against a server that declares-but-doesn't-configure the extended card, it will produce the same false failure fix: skip CARD-EXT-002 when extended card is configured #186 just eliminated for CARD-EXT-002.

Suggested work

• Tighten TestExtendedCardNotConfigured so a wrong-but-present error code fails rather than skips.
• Apply the same probe-and-skip routing to CARD-EXT-001.

Follow-up from review of #186.

[millsmillsymills]

Re-checked against main — correcting my own framing:

• TestExtendedCardNotConfigured is not on main. It exists only in PR fix: skip CARD-EXT-002 when extended card is configured #186 (still OPEN). So the skip-vs-fail behaviour I describe is a property of the unmerged PR, not current main.
• On main, no test exercises CARD-EXT-001 or CARD-EXT-002 at all, and their RequirementSpecs are structurally identical (both still operation=GET_EXTENDED_AGENT_CARD, no expected_error, agent_card.py:94 and :112). So neither produces an active false failure today — the "false-failure class" is latent in the spec, and the skip-vs-fail gap goes live only once fix: skip CARD-EXT-002 when extended card is configured #186 lands.

The substantive points stand: (1) PR #186's _run pytest.skips on a wrong-but-present error code (validate_expected_error returns non-empty on a code mismatch → skip, never fail), so a real MUST violation can go unreported; (2) -32007/EXTENDED_AGENT_CARD_NOT_CONFIGURED_ERROR mapping is correct (base.py:309).

Recommendation: fold both fixes into #186 before it merges — (a) tighten the skip heuristic so a wrong A2A error code fails rather than skips, and (b) apply the same expected_error routing to CARD-EXT-001. Keeping open as the tracker for that.