feat: wire Playwright into CI - #8558
Conversation
Branch preview✅ Deploy successful! Storybook: |
📦 Next.js Bundle Analysis for @safe-global/webThis analysis was generated by the Next.js Bundle Analysis action. 🤖 🎉 Global Bundle Size Decreased
DetailsThe global bundle is the javascript bundle that loads alongside every page. It is in its own category because its impact is much higher - an increase to its size means that every page on your website loads slower, and a decrease means every page loads faster. Any third party scripts you have added directly to your app using the If you want further insight into what is behind the changes, give @next/bundle-analyzer a try! Fifteen Pages Changed SizeThe following pages changed size from the code in this PR compared to its base branch:
DetailsOnly the gzipped size is provided here based on an expert tip. First Load is the size of the global bundle plus the bundle for the individual page. If a user were to show up to your website and land on a given page, the first load size represents the amount of javascript that user would need to download. If Any third party scripts you have added directly to your app using the Next to the size is how much the size has increased or decreased compared with the base branch of this PR. If this percentage has increased by 20% or more, there will be a red status indicator applied, indicating that special attention should be given to this. |
Coverage report for
|
St.❔ |
Category | Percentage | Covered / Total |
|---|---|---|---|
| 🟢 | Statements | 84.87% (-0.29% 🔻) |
35052/41303 |
| 🟡 | Branches | 69.75% (-0.55% 🔻) |
11443/16405 |
| 🟡 | Functions | 73.1% (-0.44% 🔻) |
5341/7306 |
| 🟢 | Lines | 86.11% (-0.29% 🔻) |
31375/36434 |
Show new covered files 🐣
St.❔ |
File | Statements | Branches | Functions | Lines |
|---|---|---|---|---|---|
| 🟢 | ... / UpsertProposer.tsx |
82.05% | 60.87% | 60% | 83.04% |
| 🟢 | ... / useRpcEndpointInfo.ts |
100% | 100% | 100% | 100% |
Show files with reduced coverage 🔻
St.❔ |
File | Statements | Branches | Functions | Lines |
|---|---|---|---|---|---|
| 🟡 | ... / utils.ts |
77.68% (-3.57% 🔻) |
62.32% (-2.9% 🔻) |
72.73% (-4.55% 🔻) |
79.17% (-3.13% 🔻) |
| 🟢 | ... / constants.ts |
58.33% (-8.33% 🔻) |
100% | 100% | 100% |
| 🟡 | ... / web3.ts |
73.17% (-26.83% 🔻) |
25% (-75% 🔻) |
50% (-50% 🔻) |
67.86% (-32.14% 🔻) |
| 🟡 | ... / utils.ts |
78.72% (-1.28% 🔻) |
52.63% | 81.82% (-2.8% 🔻) |
77.14% (-1.24% 🔻) |
| 🟢 | ... / rtkQuery.ts |
100% (+2.56% 🔼) |
94.74% (-1.26% 🔻) |
100% | 100% |
| 🟡 | ... / transactions.ts |
58.28% | 29.31% (-1.72% 🔻) |
35.71% | 60.16% |
| 🟢 | ... / useSiwe.tsx |
85% (-8.18% 🔻) |
60% (-11.43% 🔻) |
100% | 83.78% (-8.9% 🔻) |
| 🟢 | ... / index.ts |
81.25% (-6.25% 🔻) |
100% | 100% | 100% |
| 🟢 | ... / index.tsx |
100% | 71.43% (-28.57% 🔻) |
100% | 100% |
| 🟢 | ... / SeverityIcon.tsx |
100% | 80% (-20% 🔻) |
100% | 100% |
| 🟢 | ... / useAllAddressBooks.ts |
89.47% (-1.05% 🔻) |
70.37% | 80% (-5% 🔻) |
93.59% |
| 🟢 | ... / tx-list.ts |
96.61% (-1.69% 🔻) |
90.32% (-3.23% 🔻) |
93.75% (-6.25% 🔻) |
96.3% (-1.85% 🔻) |
| 🟢 | ... / index.tsx |
100% | 98.36% (-0.1% 🔻) |
100% | 100% |
| 🟢 | ... / index.tsx |
94.29% (-0.16% 🔻) |
93.75% (-0.37% 🔻) |
83.33% | 96.88% (-0.09% 🔻) |
| 🟢 | ... / ExpandableTransactionItem.tsx |
100% | 87.5% (-4.17% 🔻) |
100% | 100% |
| 🟢 | ... / index.ts |
58.33% (-5.95% 🔻) |
100% | 100% | 100% |
| 🟢 | ... / index.tsx |
93.66% (-0.38% 🔻) |
91.3% (-0.36% 🔻) |
89.19% (-0.28% 🔻) |
95.87% (-0.26% 🔻) |
| 🟡 | ... / EditProposerDialog.tsx |
56.52% (-8.18% 🔻) |
0% | 0% | 61.9% (-6.85% 🔻) |
| 🟢 | ... / useSubmitDelegation.ts |
96.97% (-0.09% 🔻) |
85.71% (+5.71% 🔼) |
100% | 96.88% (-0.09% 🔻) |
| 🟡 | ... / useTransactionType.tsx |
66.07% (-33.93% 🔻) |
37.74% (-58.1% 🔻) |
100% | 64.81% (-35.19% 🔻) |
| 🟢 | ... / index.tsx |
100% | 66.67% (-33.33% 🔻) |
100% | 100% |
| 🟢 | ... / index.tsx |
100% | 71.43% (-4.76% 🔻) |
100% | 100% |
| 🔴 | ... / index.tsx |
53.57% (-35.32% 🔻) |
0% (-33.33% 🔻) |
0% (-66.67% 🔻) |
57.69% (-30.31% 🔻) |
| 🟡 | ... / hooks.ts |
78.62% (-0.13% 🔻) |
60.81% | 78.38% | 78.83% (-0.15% 🔻) |
| 🟢 | ... / safe-migrations.ts |
97.37% (-0.07% 🔻) |
95% | 100% | 96.97% (-0.09% 🔻) |
| 🟢 | ... / index.tsx |
93.26% (-2.3% 🔻) |
79.49% (-3.01% 🔻) |
100% | 94.05% (-2.42% 🔻) |
| 🟢 | ... / useOnboardingSubmit.ts |
98.51% (-0.03% 🔻) |
93.75% (-0.13% 🔻) |
100% | 99.19% (-0.02% 🔻) |
| 🟢 | ... / index.tsx |
81.87% (-0.09% 🔻) |
56.41% | 70.27% | 83.24% (-0.09% 🔻) |
| 🟢 | ... / useMixpanel.ts |
100% | 90% (-0.24% 🔻) |
100% | 100% |
| 🟢 | ... / ImportAddressBookDialog.tsx |
97.56% (-0.03% 🔻) |
86.67% | 88.24% | 98.63% (-0.02% 🔻) |
| 🟢 | ... / utils.ts |
97.06% (-0.31% 🔻) |
88.89% (-2.78% 🔻) |
100% | 100% |
| 🟢 | ... / index.tsx |
98.25% (-0.31% 🔻) |
89.29% (-0.37% 🔻) |
100% | 98.08% (-0.36% 🔻) |
| 🟢 | ... / AddToWorkspaceButton.tsx |
94.29% (-0.59% 🔻) |
86.67% | 100% | 96.97% (-0.33% 🔻) |
| 🟢 | ... / MemberRowActionsMenu.tsx |
90.48% (-0.43% 🔻) |
100% | 66.67% | 90% (-0.48% 🔻) |
| 🟢 | ... / index.tsx |
86.96% (-0.28% 🔻) |
56.25% (-8.04% 🔻) |
33.33% | 92.86% (-0.17% 🔻) |
| 🟢 | ... / RequestToAddButton.tsx |
85.25% (-0.69% 🔻) |
76.47% | 71.43% | 89.29% (-0.54% 🔻) |
| 🟢 | ... / useInitSafeCoreSDK.ts |
91.18% (-8.82% 🔻) |
87.5% (-2.5% 🔻) |
75% (-25% 🔻) |
90.63% (-9.38% 🔻) |
| 🔴 | ... / PendingRequestsTable.tsx |
56.72% (-30.61% 🔻) |
21.05% (-26.32% 🔻) |
42.86% (-35.71% 🔻) |
58.73% (-32.31% 🔻) |
| 🔴 | ... / index.tsx |
35.71% (-57.14% 🔻) |
0% (-75% 🔻) |
0% (-90% 🔻) |
45.45% (-50% 🔻) |
| 🟡 | ... / AddManually.tsx |
61.36% (-7.53% 🔻) |
33.33% | 11.11% (-33.33% 🔻) |
63.41% (-5.63% 🔻) |
Test suite run success
7600 tests passing in 875 suites.
Report generated by 🧪jest coverage report action from 77a87c5
…pec (#8550) Design QA follow-up on WA-3121 (#8544): - green status badge uses the brand success pairing (theme-flipping, #EFFAF1/#028D4C light, #1F2920/#388E3C dark), scoped to size=status - dark-mode yellow status badge pinned to the DS spec (#301D02/#EAB308, dot #FACC15); light mode and the shared warning tokens unchanged - tooltip copy: 'N signers can't enable 2FA because wallet login doesn't support it' Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: tx queue header * fix: signature badge * fix: paddings in tx details * fix: tx details bg * fix: audit log * fix: restore no prefix for address * fix: tx links * fix: notes * fix: all actions header * fix: multi-send actions * fix: tx action spaces * fix: tx status chip * fix: HN log in * fix: badge colors * fix: disclaimer * fix: safe update * chore: ai comments * fix: merge conflict
…87 (#8477) Bumps [anthropics/claude-code-action](https://github.com/anthropics/claude-code-action) from 1.0.183 to 1.0.187. - [Release notes](https://github.com/anthropics/claude-code-action/releases) - [Commits](anthropics/claude-code-action@be7b93b...1623c36) --- updated-dependencies: - dependency-name: anthropics/claude-code-action dependency-version: 1.0.187 dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 7.2.0 to 7.3.0. - [Release notes](https://github.com/docker/build-push-action/releases) - [Commits](docker/build-push-action@f9f3042...53b7df9) --- updated-dependencies: - dependency-name: docker/build-push-action dependency-version: 7.3.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.2.0 to 7.0.0. - [Release notes](https://github.com/actions/setup-python/releases) - [Commits](actions/setup-python@a309ff8...5fda3b9) --- updated-dependencies: - dependency-name: actions/setup-python dependency-version: 7.0.0 dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ll toJSON (#8533) * chore: pass enumerated secrets to CI actions instead of dumping toJSON(secrets) * feat: script for resolving web build envs * chore: pass CI secrets as per-step env vars instead of JSON * chore: add missing env vars * test: add tests for missing env vars + enumerated manifest with keys listed * test: add non-`NEXT_PUBLIC_*` keys to the test to make it more robust * fix: add a missing env var for the Storybook stories
Tim (tmjssz)
left a comment
There was a problem hiding this comment.
Great, thanks for taking the initiative!
What it solves
Resolves:
The Playwright framework in
apps/web/e2e/existed but nothing in CI ran it —web-e2e-smoke.yml,web-e2e-full-ondemand.ymlandweb-argos-e2e.ymlall invoke the Cypress action. Every Playwright guarantee rested on someone running it locally, and the suite had rotted accordingly:safe-header-info— the testid the dashboard smoke test waits on was deleted a month after the locator was written (refactor(web): dead code cleanupremoved the dead MUISafeHeaderInfocomponent). The test had been timing out at 16s.address-book-toggle— a regression spec referenced a testid the app never shipped; the PR that added the spec only addedaddress-itemandcontact-group-header.private-key-input—getByTestId(...).locator('input')was correct under MUI'sTextField(testid on a wrapper) and wrong after the shadcn migration (testid on the bare<input>), so every wallet-connected spec failed at fixture setup.credentials.spec.tsand the wallet-gated regression specs were mutually exclusive: the former assertsgetWalletCredentials(undefined)throws, which only happens whenCYPRESS_WALLET_CREDENTIALSis unset, while the latter require it set. They could never both pass in one run.e2e/tsconfig.jsoncould not compile its own fixture (TS7016, leavingTERMS_VERSIONan implicitany).Separately,
AGENTS.mddeclares Playwright the only home for new tests and Cypress legacy, but no migration had started and there was no plan for the 153 remaining Cypress specs.How this PR fixes it
Three things, in order of importance:
1. Wires Playwright into CI. New
.github/actions/playwrightcomposite action mirroring the Cypress one (yarn→build→ browser install → run), plusweb-pw-smoke.yml(@smoke|@api, blocking, every PR) andweb-pw-full-ondemand.yml(@regression, dispatch/schedule). Cypress workflows are untouched — both suites run in parallel, which is what makes staged deletion safe later.Notably the action does not set
NODE_ENV=cypress. That value's only real effect is skipping SRI generation, which Cypress needs because it injects<script>tags; Playwright seeds via CDP (addInitScript), which never touches document HTML. Verified against a real SRI-enabled build (224 integrity attrs, 290-chunk manifest) — zero integrity/CSP errors. So this tests a production-shaped build.2. Fixes the rot. Restored
safe-header-infoon the current Safe-identity block (the original component is gone; this is its counterpart, wrapping the same icon + address + balance), addedaddress-book-toggleto the real caret button, corrected theprivate-key-inputlocator, made the credentials test deterministic with save/restore around the one assertion that legitimately readsprocess.env, and fixed the e2e tsconfig (allowJs, droppedrootDir, added a@/*alias so the fixture import matches the other six consumers).3. Documents the migration as an executable programme. Extends the existing
CYPRESS_MIGRATION_GUIDE.md(already the docAGENTS.mdpoints agents at) with a 14-stage order, an 8-step per-stage loop, prerequisites, and a Known-hazards table using the real failures above as evidence. AddsMIGRATION_STATUS.md— a generated inventory of all 153 specs (122 in scope, 31 Argosvisual/out of scope), every one assigned to a stage, with wallet/mock flags. It records that nothing is genuinely migrated yet:dashboard.spec.tsasserts 3 things wheresmoke/dashboard.cy.jsasserts 6.How to test it
Affected flows
data-testidadded to the identity block. Attribute only, no behaviour or markup change.data-testidadded to the address-book caret button. Attribute only.apps/web/**orpackages/**.Blast radius
AddressInput(src/components/common/AddressInput/index.tsx) — shared by every recipient field across tx flows. One added attribute; the caret only renders whenonOpenListClickis passed, i.e. when there are visible options.SafeSelectorTriggerContent— single render site (SafeSelectorDropdown/index.tsx:200), verified sogetByTestIdcan't trip strict mode. No__snapshots__in that tree, so no snapshot churn.apps/web/package.json—pw:cigrep widened from@smoketo@smoke|@api(CI now also runs the 12 API tests); addedpw:test:regression.e2e/tsconfig.json— governs Playwright's TS transpilation for all e2e files. The repo'stsc --noEmitusesapps/web/tsconfig.json, whose**/*.tsinclude already coverse2e/; both were re-run green.packages/**changes, so no mobile impact.Risks / not checked
address-book-toggleships without a unit guard. The two caret-toggle tests written forAddressBookInput/index.test.tsxare not in the branch — so a new app testid is consumed only by a@regressionspec that the blocking PR workflow does not run. This is precisely the hazard this PR documents. Recommend restoring before merge.next devon:3000; CI builds and serves the staticout/on:8080. I replicated that locally (real build +serve+CI=true) and it passed 16/16, but the composite action itself has never executed on a runner.balances.spec.tsis flaky under CI'sfullyParallel. Reproducible and diagnosed: serial is 16/16 green in 7.2s; parallel intermittently overruns the 10sexpecttimeout. Ruled out by experiment — not SRI (reproduces both ways), not static-vs-dev server (both), not cold start, not@apiload (fails with@smokealone).retries: 2means it lands as "flaky" and the job still exits 0, so CI will be green-but-noisy. Left unfixed — the fix belongs at the readiness level and is a call for the team._BEAMER_*keys without thePRODUCT_IDsuffix Beamer actually reads, and setsupdates: true— the consent that loads the widget. Cypress guards it twice (product-ID keys andblockBeamer(), whose comment says the popup "covers onboarding buttons"). Documented and deliberately deferred: it bites from Stage 5 (sidebar.pages.js) onward; stages 1–4 are clear.@regressiontests exist and 3 need the wallet secret; runner-side signing is unexercised.visual/specs stay and importmain.page.js, so the page-object tree survives until a separate@argos-ci/playwrightproject re-baselines them.Visual summary
CI coverage — before and after
flowchart TB subgraph Before["Before — Playwright unguarded"] PR1[Pull request] --> CY1[web-e2e-smoke.yml<br/>Cypress action] CY1 --> C1[153 Cypress specs] PW1[["apps/web/e2e<br/>20 Playwright tests"]] PW1 -.->|"no workflow runs this"| ROT["silent rot:<br/>safe-header-info<br/>address-book-toggle<br/>private-key-input"] end subgraph After["After — both suites guarded"] PR2[Pull request] --> CY2[web-e2e-smoke.yml<br/>Cypress action] PR2 --> PWCI[web-pw-smoke.yml<br/>playwright action] CY2 --> C2[153 Cypress specs] PWCI --> S["@smoke + @api<br/>16 tests, blocking"] DISP[workflow_dispatch<br/>+ schedule] --> PWFULL[web-pw-full-ondemand.yml] PWFULL --> R["@regression<br/>20 tests"] end Before ==>|this PR| AfterThe per-stage migration loop the docs now mandate
flowchart LR A[Classify<br/>8 questions] --> B[Write spec<br/>12-step format] B --> C{"--repeat-each=10<br/>10/10?"} C -->|no| B C -->|yes| D[Full Playwright<br/>suite green] D --> E[Prove Cypress spec<br/>still passes] E --> F[Delete Cypress spec<br/>+ update tracker] F --> G{"Cypress suite<br/>still green?"} G -->|no| H["shared page-object<br/>broke — fix"] H --> G G -->|yes| I[Stage merged]Checklist
NEXT_PUBLIC_IS_TEST_E2Ekeeps Datadog RUM off in test builds, as beforesafe-header-infoguarded inSafeSelectorTriggerContent.test.tsx;address-book-toggleguard is missing, see RisksCLA signature
With the submission of this Pull Request, I confirm that I have read and agree to the terms of the Contributor License Agreement.