|
| 1 | +# Open Questions |
| 2 | + |
| 3 | +## Product |
| 4 | + |
| 5 | +- Should v1 run as a CLI only or include a web dashboard? |
| 6 | +- Should reports be Markdown-only, JSON-only, spreadsheet-compatible, or all three? |
| 7 | +- Should screenshots be embedded in report output or linked as local files? |
| 8 | +- Should the agent classify issues as bug, UX issue, feature gap, performance issue, or blocker? |
| 9 | +- Should the agent support test plans written in Markdown? |
| 10 | + |
| 11 | +## Browser Workflow |
| 12 | + |
| 13 | +- Which workflows are approved for default testing? |
| 14 | +- Should the agent auto-discover navigation or follow an explicit route list? |
| 15 | +- Should it use visible text selectors, test IDs, or a hybrid selector strategy? |
| 16 | +- Should mobile viewport testing be part of MVP? |
| 17 | +- Should console/network monitoring be enabled by default? |
| 18 | + |
| 19 | +## Safety |
| 20 | + |
| 21 | +- Which actions are considered destructive for the target app? |
| 22 | +- Should archive/restore be allowed by default while permanent delete is blocked? |
| 23 | +- Should the agent require confirmation before submitting forms? |
| 24 | +- What should happen if test account sign-in fails? |
| 25 | +- Should the agent mask email, phone, or customer data in reports? |
| 26 | + |
| 27 | +## Technical |
| 28 | + |
| 29 | +- Where should screenshots be stored? |
| 30 | +- Should traces be captured for every run or only on failure? |
| 31 | +- Should output include machine-readable bug IDs? |
| 32 | +- Should reports be deterministic for CI usage? |
| 33 | +- What timeout and retry policy should be used per step? |
| 34 | + |
| 35 | +## Evaluation |
| 36 | + |
| 37 | +- What fixture apps should be used for regression testing? |
| 38 | +- How do we score whether the agent found the correct bug? |
| 39 | +- How do we test that the agent avoids destructive actions? |
| 40 | +- How do we compare agent reports across versions? |
0 commit comments