autonomous-qa 0.4.4: Workers-pool-safe vitest integration, exec runner threading, warm-isolate replay - #11
Open
nathanclevenger wants to merge 7 commits into
Open
autonomous-qa 0.4.4: Workers-pool-safe vitest integration, exec runner threading, warm-isolate replay#11nathanclevenger wants to merge 7 commits into
nathanclevenger wants to merge 7 commits into
Conversation
…nomous-qa/assert; describeConformance
@cloudflare/vitest-pool-workers runs test files in workerd and externalizes
node_modules, so `autonomous-qa/vitest`'s top-level `import { expect } from
'vitest'` failed to link there ("The requested module 'vitest' does not
provide an export named 'expect'"), and consumers running their gate against
a real Durable Object had to call gradePinned by hand.
- src/assert.ts (new export `autonomous-qa/assert`): assertConforms,
assertGradeAtLeast, conformance(), gradeAtLeast() — the same verdicts and
messages as the matchers, no vitest import at all. Loads wherever the core
loads.
- src/vitest.ts: no value import of 'vitest'. Matchers register via
registerConformanceMatchers(expect) → vitest globals → a lazy
import('vitest') at module load that is caught where it cannot resolve
(matchersReady exposes the outcome). describeConformance({ target, spec,
expectedDigest, baseOrigin }) expands every pinned requirement into its own
`it` (one grading pass in beforeAll; the digest is checked before any
probe fires), taking the vitest API from an explicit argument or globals
and throwing a pointed error otherwise.
- package.json: 0.4.0, exports ./assert, vitest peer marked optional.
- tests: assert.test.ts (includes a source guard that vitest.ts never value-
imports 'vitest'), describe-conformance.test.ts (explicit-API shape, the
Workers-pool form; matcher registration idempotence; the no-API error).
- README: "Vitest — Node projects and the Cloudflare Workers pool".
The skill text (skill/SKILL.md) is canon-synced from axp.org.ai and is
updated there separately.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eclared suites in-process
A card-declared executable suite (interfaces.testSuite, runner
api.qa/vitest@1) was judged runner-unavailable by grade(), gradePinned(),
the vitest matchers/describeConformance and the CLI verify/dev/grade verbs,
because none of them passed an execRunner to observeTarget — only
`autonomous-qa vitest` and the hosted verdict executed suites.
- local.ts: GradeOpts.execRunner threaded into the observer options. Default:
in Node, localExecRunner() using the target's own fetch (the in-memory
fetcher for handler targets, so a suite's egress to the target never
touches a socket); inside workerd (navigator.userAgent
'Cloudflare-Workers', where the local runner's data: imports cannot
resolve) the typed runner-unavailable naming the fix:
workerLoaderExecRunner(env.SUITE_LOADER, { outbound: createOutboundGateway(fetch) }).
defaultExecRunner and RUNNER_UNAVAILABLE_WORKERD exported.
- cli: verify, dev, grade pass localExecRunner() explicitly.
- tests: local-exec.test.ts (declared suite passes through grade() by
default; an explicit runner is used verbatim; the workerd default is the
typed unavailable). The declared-suite fixture moves to
test/suite-fixture.ts and is shared with vitest-suite-interface.test.ts.
- 0.4.1.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…est shim) workerLoaderExecRunner keys the isolate by content hash so the same bytes warm-reuse a compiled isolate — but an ES module evaluates once per isolate, so the second run of the same digest re-imported ./suite-tests.mjs without re-executing its top-level describe/it calls: the new harness had nothing registered and capability coverage collapsed (seen from sdb's pool gate: 18/18 on the first run, 5/18 on the second). The old shim also bound the run registry at evaluation, so even the recorded calls would have landed in the first run's harness. - vitestShimSource: every export resolves the CURRENT run's harness at call time; top-level registrations (depth 0) are recorded; __replay() re-issues them into the new harness (nested registrations re-happen inside the replayed describe bodies and are not recorded again); __beginRun() reports warmth from the shim instance itself — one per module graph, i.e. per isolate — never from globalThis. - entrySource: installs the document-form globals from the shim (not the harness api directly) and replays when __beginRun() says warm. - dataModuleUrl: escape quotes — these URLs sit inside quoted import specifiers and the shim now contains string literals. - test/hosted-warm-isolate.test.ts: a loader keyed by id (same evaluated module graph) runs a suite three times, module and document forms, and every run registers and passes the same tests from one isolate spin. - 0.4.2. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…an ambient wrapper Seen from `autonomous-qa verify <live url>`: the executed suite's tests died with "Maximum call stack size exceeded" and, on concurrent runs, "describe is not defined" — while the same suite passed standalone. - localExecRunner runs are queued: it swaps process-wide state (fetch, Math.random, the document-form globals) for a run, so overlapping runs saw each other's harness and the first to finish deleted the globals the second was still importing against. A.8.6.4 wants sequential execution anyway; it is now a property of the runner. - The gated fetch's real fetch is the platform fetch captured once at module load, not the ambient fetch at run start: a host wrapper that resolves globalThis.fetch lazily made gate → wrapper → gate recurse. - test/local-runner-reentrancy.test.ts pins both. - 0.4.3. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…y globalThis.fetch)
discovery hands the executable-suite runner `observer.transportFetcher` as
the gate's real fetch. The default transport resolved `globalThis.fetch`
lazily, and during a run that global IS the gated fetch — so every suite
test recursed gate → transport → gate ("Maximum call stack size exceeded"
from `autonomous-qa verify <live url>`, while the same suite passed
standalone and hosted). The transport now binds the platform fetch once at
module load. Pinned by a test that runs a suite through a default Observer's
transport. 0.4.4.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 331ae55b9e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+421
to
+424
| const warm = SHIM.__beginRun() | ||
| if (HAS_MODULE) await import('./suite-module-impl.mjs') | ||
| const ns = await import('./suite-tests.mjs') | ||
| if (warm) SHIM.__replay() |
There was a problem hiding this comment.
When the same artifact digest is run for another target, environment, or seed, workerLoaderExecRunner reuses the existing module graph, but SEED, SANDBOX, EXPORT_NAME, and the entire suite:env module were compiled from the first request. This replay therefore probes the first origin with its vars and seed; more seriously, an isolate first created with sandbox: true continues allowing mutating requests during a later non-sandbox run. Make these values request-scoped or include every embedded input in the loader identity before treating the isolate as warm.
Useful? React with 👍 / 👎.
Comment on lines
+424
to
427
| if (warm) SHIM.__replay() | ||
| if (EXPORT_NAME !== null) { | ||
| const fn = ns[EXPORT_NAME] | ||
| if (typeof fn !== 'function') throw new Error('the card names export "' + EXPORT_NAME + '", but the pinned module has no such function export') |
There was a problem hiding this comment.
For a module artifact with exportName, the first invocation of that export registers its tests through the shim and records those calls. On every warm invocation this block replays those registrations and then calls the export again, so the second run registers each test twice, the third three times, and so on. This can duplicate API operations, alter results, and eventually trip the combined-test cap; exported registration functions should be invoked once per run without replaying their prior calls.
Useful? React with 👍 / 👎.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proven by @dotdo/sdb: pool gate executes its card-declared suite through a real Worker Loader; hosted verdict 18/18 operations covered.
🤖 Generated with Claude Code