Enable testTagsAsResourceId and consolidate screen identifiers into verify skill by JorgeMucientes · Pull Request #15418 · woocommerce/woocommerce-android

JorgeMucientes · 2026-02-23T19:56:30Z

Description

Three changes to improve AI agent reliability when verifying UI changes on-device:

1. Enable testTagsAsResourceId in WooTheme

Adds Modifier.semantics { testTagsAsResourceId = true } to WooTheme, which makes all Compose testTag() values visible as resource IDs in the accessibility tree. Without this, Compose test tags (e.g. dashboard_stats_card, POS tags) were invisible to mobile_list_elements_on_screen, the primary tool agents use to detect and interact with UI elements. This is a one-line semantic change with no visual or layout impact.

2. Consolidate screen identifiers into the verify-on-device skill

Moves the screen identifiers reference from a standalone file (.agents/references/screen-identifiers.md) into the verify-on-device skill where it is actually consumed. This gives agents a single source of truth, the skill now contains comprehensive screen detection identifiers, navigation flows, dialog handling, and Compose test tag references inline.

3. Add device preparation steps to the verify skill

Adds three new steps to reduce flakiness and interference during agent verification:

Disable animations via ADB (animator_duration_scale, transition_animation_scale, window_animation_scale set to 0) so element detection is not disrupted by transitions.
Disable LeakCanary by writing enable_leak_canary=false to developer.properties before building, preventing leak detection dialogs from blocking the UI.
Launch MainActivity explicitly via adb am start instead of mobile_launch_app, ensuring the app always opens to the main screen.

Test Steps

Build and install the debug APK on an emulator
Open any Compose screen that uses testTag() (e.g. Dashboard)
Run: adb shell uiautomator dump /sdcard/ui.xml and adb pull /sdcard/ui.xml and grep dashboard_stats_card ui.xml. You should see the test tag appear as a resource-id in the UI hierarchy. Without this change, it would not appear.
Verify the app renders normally (no visual regressions from the Box wrapper)
Review .agents/skills/verify-on-device/SKILL.md and confirm the WooCommerce Navigation Reference section contains complete screen identifiers and navigation flows

Images/gif

N/A

I have considered if this change warrants release notes and have added them to RELEASE-NOTES.txt if necessary. Use the [Internal] label for non-user-facing changes.

… testTagsAsResourceId Move the screen identifiers reference from a standalone document (.agents/references/screen-identifiers.md) into the verify-on-device skill (SKILL.md), so agents have all navigation context in one place when performing device verification. This also adds two missing dialog entries (Store Name and Create Test Order) and expands the navigation reference with per-screen element tables and step-by-step flows. Enable testTagsAsResourceId in WooTheme so that Compose test tags are exposed as resource IDs in the accessibility tree. This allows UI automation tools (Appium, UI Automator, mobile-mcp) to discover Compose elements by their test tags, which is essential for reliable agent-driven device verification.

Disable animations, LeakCanary, and launch MainActivity explicitly to reduce flakiness and interference during agent verification.

Resolve conflicts in SKILL.md step numbering, incorporating trunk's new API Mocks step alongside our device preparation steps.

wpmobilebot · 2026-03-03T19:13:47Z

📲 You can test the changes from this Pull Request in WooCommerce Android by scanning the QR code below to install the corresponding build.

	App Name	WooCommerce Android
	Platform	📱 Mobile
	Flavor	Jalapeno
	Build Type	Debug
	Build Number	`730`
	Version	`24.2`
	Application ID	`com.woocommerce.android.prealpha`
	Commit	`5fd8cff`
	Installation URL	7escsbdo204u8

Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

codecov-commenter · 2026-03-03T22:00:36Z

Codecov Report

❌ Patch coverage is 0% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 39.43%. Comparing base (6446d7d) to head (5fd8cff).
⚠️ Report is 407 commits behind head on trunk.

Files with missing lines	Patch %	Lines
.../com/woocommerce/android/ui/compose/theme/Theme.kt	0.00%	5 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##              trunk   #15418   +/-   ##
=========================================
  Coverage     39.43%   39.43%           
- Complexity    11121    11122    +1     
=========================================
  Files          2246     2246           
  Lines        128532   128536    +4     
  Branches      17968    17968           
=========================================
+ Hits          50682    50685    +3     
- Misses        72675    72677    +2     
+ Partials       5175     5174    -1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

irfano

Thanks for improving this skill, Jorge! I've added some feedback, please take a look.

irfano · 2026-03-03T21:58:13Z

.agents/skills/verify-on-device/SKILL.md

+Reviews List:      Menu tab → tap "Reviews"
+Analytics Hub:     Dashboard → tap "See analytics" or Menu tab → "Analytics"
+Customer List:     Menu tab → tap "Customers"
+Blaze Campaigns:   Menu tab → tap "Blaze Campaigns"


The menu button is just "Blaze", not "Blaze Campaigns". Let's update it.

irfano · 2026-03-04T09:24:17Z

.agents/skills/verify-on-device/SKILL.md

+Payments Hub:      Menu tab → tap "Payments"
+Coupon List:       Menu tab → tap "Coupons"
+Reviews List:      Menu tab → tap "Reviews"
+Analytics Hub:     Dashboard → tap "See analytics" or Menu tab → "Analytics"


We don't have "Analytics" at Menu.

I think this should be like this:

Suggested change

Analytics Hub: Dashboard → tap "See analytics" or Menu tab → "Analytics"

Analytics Hub: Dashboard → tap "View all store analytics"

irfano · 2026-03-04T11:24:36Z

.agents/skills/verify-on-device/SKILL.md

+|--------|----------|----------|
+| Main Settings | `MainSettingsFragment` | Menu → "Settings" |
+| Privacy Settings | `PrivacySettingsFragment` | Settings → "Privacy settings" |
+| Beta Features | `BetaFeaturesFragment` | Settings → "Beta features" |


irfano · 2026-03-04T11:32:48Z

.agents/skills/verify-on-device/SKILL.md

+
+Settings:          Menu tab → tap "Settings" (opens AppSettingsActivity)
+Privacy Settings:  Settings → tap "Privacy settings"
+Beta Features:     Settings → tap "Beta features"


Same comment here. The screen is called "Experimental Features" in the UI.

Suggested change

Beta Features: Settings → tap "Beta features"

Experimental Features: Settings → tap "Experimental features"

.agents/skills/verify-on-device/SKILL.md

irfano · 2026-03-04T12:06:02Z

.agents/skills/verify-on-device/SKILL.md

+| Privacy Settings | `PrivacySettingsFragment` | Settings → "Privacy settings" |
+| Beta Features | `BetaFeaturesFragment` | Settings → "Beta features" |
+| Developer Options | `DeveloperOptionsFragment` | Settings → "Developer options" (debug only) |
+| Notifications | `NotificationSettingsFragment` | Settings → "Notifications" |


It's actually tricky to open NotificationSettingsFragment since the user has to manually change the default notification sound to open our notification screem. Currently, this button just opens the OS notification settings. I think it's fine to omit mentioning NotificationSettingsFragment. We can use this instead:

Suggested change

| Notifications | `NotificationSettingsFragment` | Settings → "Notifications" |

| Manage Notifications | OS Notification Settings | Settings → Manage Notifications |

Alternatively, we could just drop this line entirely.

wpmobilebot · 2026-03-06T13:09:18Z

Version 24.3 has now entered code-freeze, so the milestone of this PR has been updated to 24.4.

JorgeMucientes · 2026-03-08T16:55:47Z

Thanks for the feedback @irfano
I've addressed all the suggestions.
Ready for another round 👍🏼

irfano

Thanks for addressing my feedback. I’ve just left two more minor suggestions, but I’m approving. LGTM! 👍🏻

.agents/skills/verify-on-device/SKILL.md

Co-authored-by: Irfan Ömür <irfan.omur@automattic.com>

Add screen identifiers reference for agent device verification

bcdc53b

RafaelKayumov mentioned this pull request Mar 2, 2026

Add AI agent E2E verification capabilities woocommerce/woocommerce-ios#16752

Merged

1 task

JorgeMucientes changed the title ~~Add screen identifiers reference for agent device verification~~ Enable testTagsAsResourceId and consolidate screen identifiers into verify skill Mar 3, 2026

Add device preparation steps to verify-on-device skill

6c83abf

Disable animations, LeakCanary, and launch MainActivity explicitly to reduce flakiness and interference during agent verification.

JorgeMucientes requested review from AdamGrzybkowski, hichamboushaba and irfano March 3, 2026 19:02

Merge trunk into issue/agent-screen-identifiers

2129ef0

Resolve conflicts in SKILL.md step numbering, incorporating trunk's new API Mocks step alongside our device preparation steps.

JorgeMucientes marked this pull request as ready for review March 3, 2026 20:58

JorgeMucientes added this to the 24.3 milestone Mar 3, 2026

JorgeMucientes added the type: task An internally driven task. label Mar 3, 2026

irfano self-assigned this Mar 3, 2026

irfano requested changes Mar 4, 2026

View reviewed changes

wpmobilebot modified the milestones: 24.3, 24.4 Mar 6, 2026

JorgeMucientes added 5 commits March 8, 2026 17:47

Update Notifications row to Manage Notifications with OS settings

ece621d

Rename Beta Features to Experimental Features in nav path section

858bb45

Fix Analytics Hub nav path to use correct menu label

46d72c2

Rename Blaze Campaigns to Blaze in menu nav path

8e9b18b

Rename Beta Features to Experimental features in settings table

fb16ddf

JorgeMucientes requested a review from irfano March 8, 2026 16:55

irfano approved these changes Mar 9, 2026

View reviewed changes

.agents/skills/verify-on-device/SKILL.md Outdated Show resolved Hide resolved

.agents/skills/verify-on-device/SKILL.md Outdated Show resolved Hide resolved

JorgeMucientes and others added 2 commits March 11, 2026 08:38

Properly name Blaze more menu CTA

f28bc0b

Co-authored-by: Irfan Ömür <irfan.omur@automattic.com>

Update description to analytics hub entry point

5fd8cff

Co-authored-by: Irfan Ömür <irfan.omur@automattic.com>

JorgeMucientes enabled auto-merge March 11, 2026 07:40

JorgeMucientes merged commit 40a3f3c into trunk Mar 11, 2026
16 checks passed

JorgeMucientes deleted the issue/agent-screen-identifiers branch March 11, 2026 07:57

	Analytics Hub: Dashboard → tap "See analytics" or Menu tab → "Analytics"
	Analytics Hub: Dashboard → tap "View all store analytics"

	\| Beta Features \| `BetaFeaturesFragment` \| Settings → "Beta features" \|
	\| Experimental features \| `BetaFeaturesFragment` \| Settings → "Experimental features" \|

	Beta Features: Settings → tap "Beta features"
	Experimental Features: Settings → tap "Experimental features"

	\| Notifications \| `NotificationSettingsFragment` \| Settings → "Notifications" \|
	\| Manage Notifications \| OS Notification Settings \| Settings → Manage Notifications \|

Conversation

JorgeMucientes commented Feb 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Test Steps

Images/gif

Uh oh!

wpmobilebot commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

irfano left a comment

Choose a reason for hiding this comment

Uh oh!

irfano Mar 3, 2026

Choose a reason for hiding this comment

Uh oh!

irfano Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

irfano Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

irfano Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

irfano Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

wpmobilebot commented Mar 6, 2026

Uh oh!

JorgeMucientes commented Mar 8, 2026

Uh oh!

irfano left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

JorgeMucientes commented Feb 23, 2026 •

edited

Loading

wpmobilebot commented Mar 3, 2026 •

edited

Loading

codecov-commenter commented Mar 3, 2026 •

edited

Loading