Skip to content

Commit f87fee9

Browse files
BAKUGOS1BAKUGOS1
committed
feat: autonomous explorer, real detectors, selector healing, safety guard v2, new browser actions, API interceptor, ARCHITECTURE.md, README overhaul
1 parent fddb4fe commit f87fee9

17 files changed

Lines changed: 1176 additions & 389 deletions

‎ARCHITECTURE.md‎

Lines changed: 211 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,211 @@
1+
# QaAgent — System Architecture & Design Specification
2+
3+
This document details the engineering architecture, data flows, components, and design decisions of **QaAgent**, a professional-grade TypeScript + Playwright QA automation platform.
4+
5+
---
6+
7+
## 🗺️ System Topology & Execution Modes
8+
9+
QaAgent is built around a single, highly-instrumented local browser automation engine, supporting two reasoning interfaces:
10+
11+
```mermaid
12+
flowchart TB
13+
subgraph Input ["Input Layer"]
14+
A[CLI Arguments / JSON Task File] --> B[loadConfig / loadTask]
15+
end
16+
17+
subgraph Interface ["Execution Modes"]
18+
B --> C[Codex Mode / local-first]
19+
B --> D[Groq Mode / API-driven]
20+
end
21+
22+
subgraph Core ["Local Browser Engine"]
23+
C --> E[BrowserAgent]
24+
D --> E
25+
E --> F[Playwright Browser Context]
26+
F --> G[ConsoleListener]
27+
F --> H[NetworkListener + API Interceptor]
28+
end
29+
30+
subgraph Brain ["QA Intelligence"]
31+
E --> I[Page Analyzer]
32+
I --> J[BrowserState JSON]
33+
J --> K[QA Engine]
34+
K --> L[Detectors]
35+
K --> M[Declarative Playbooks]
36+
end
37+
38+
subgraph Output ["Report Generation"]
39+
L --> N[Report Writer]
40+
M --> N
41+
N --> O[Zero-Dep OOXML Excel Report]
42+
N --> P[Markdown & JSON Debug Logs]
43+
end
44+
45+
classDef core fill:#f9f,stroke:#333,stroke-width:2px;
46+
classDef brain fill:#bbf,stroke:#333,stroke-width:2px;
47+
classDef output fill:#bfb,stroke:#333,stroke-width:2px;
48+
class E,F,I core;
49+
class K,L,M brain;
50+
class O,P output;
51+
```
52+
53+
### 1. Codex / no-API Mode
54+
- **Rationale**: For local-first development where credentials/auth should never leave the machine.
55+
- **Workflow**: Codex (running within the chat client) acts as the reasoning engine. The local `codex-driver` handles page initialization, explicit steps, autonomous exploration, and local detectors, writing rich evidence and reports locally.
56+
57+
### 2. Groq / API Mode
58+
- **Rationale**: Fully autonomous standalone CLI tool loop.
59+
- **Workflow**: A tool-use loop executes against a Groq model (e.g. `gpt-oss-120b`). The model chooses tool calls (e.g. click, fill, scroll, hover), and the browser agent executes them, handling recoveries and checking safety guards on each action.
60+
61+
---
62+
63+
## 📂 Source Code Mapping
64+
65+
The codebase is organized into modular directories under `agent/src/` to isolate automation, reasoning, intelligence, and reporting:
66+
67+
```text
68+
agent/src/
69+
├── api-agent/ # Groq tool loop definitions and client orchestration
70+
│ ├── groq-client.ts
71+
│ ├── groq-tool-definitions.ts
72+
│ └── groq-tool-loop.ts
73+
├── browser/ # Playwright orchestration and DOM analyzer
74+
│ ├── actions.ts # Maps command strings to browser method execution
75+
│ ├── browser-agent.ts # Unified browser context, state cache, actions
76+
│ ├── console-listener.ts
77+
│ ├── login-runner.ts # Secure credential autofill and validation
78+
│ ├── network-listener.ts# Collects network errors & intercepting API payloads
79+
│ ├── page-analyzer.ts # Computes accessible DOM representation
80+
│ ├── recorder.ts # Auditing browser actions for coverage verification
81+
│ └── selector-healer.ts # Multi-strategy selector repair
82+
├── codex-agent/ # Codex mode driver and autonomous exploration
83+
│ ├── autonomous-explorer.ts # Navigates origins, tests pages/forms autonomously
84+
│ ├── codex-driver.ts
85+
│ ├── codex-report-helper.ts
86+
│ └── codex-task-runner.ts
87+
├── data/ # Indian-style mock CRM lead data generators
88+
│ └── lead-data.ts
89+
├── memory/ # JSON file-backed local persistence layer
90+
│ ├── selectors-memory.ts
91+
│ ├── sites-memory.ts
92+
│ └── test-history.ts
93+
├── qa/ # QA profiles, detectors, and playbooks
94+
│ ├── detectors/ # Specialized DOM auditors
95+
│ │ ├── accessibility-detector.ts
96+
│ │ ├── form-detector.ts
97+
│ │ ├── performance-detector.ts
98+
│ │ └── table-detector.ts
99+
│ ├── playbooks/ # Scope checklists
100+
│ ├── checks.ts
101+
│ ├── coverage.ts # Formulates Pass/Partial/Fail based on action success
102+
│ ├── flaky-rules.ts
103+
│ ├── issue-detector.ts # Aggregator for all detectors
104+
│ ├── playbook-runner.ts
105+
│ ├── priority-rules.ts
106+
│ ├── qa-engine.ts
107+
│ ├── risk-rules.ts
108+
│ └── severity.ts
109+
├── reports/ # Report generation templates
110+
│ ├── excel.ts # Hand-coded OOXML ZIP compiler
111+
│ ├── json.ts
112+
│ ├── markdown.ts
113+
│ └── report-writer.ts
114+
├── shared/ # Interfaces, utils, and safety guards
115+
│ ├── safety-guard.ts # Action-filtering firewall
116+
│ ├── types.ts
117+
│ └── utils.ts
118+
└── config.ts # Local settings parser
119+
```
120+
121+
---
122+
123+
## 🛠️ Key Engineering Components
124+
125+
### 1. BrowserState Extractor (`page-analyzer.ts`)
126+
Instead of feeding raw HTML or a full screenshot to the agent, the Page Analyzer compiles a highly structured **accessible DOM map** including:
127+
- **Clickable Elements**: Evaluates focusable items, computing unique CSS selectors, roles, text names, tags, and coordinates. Every clickable element gets a simple integer index (`0-99`) for Groq tool execution.
128+
- **Form Fields**: Structures inputs, labels, placeholders, validation hints, and submit associations.
129+
- **Tables**: Parses rows, headers, cell matrices, and paginator elements.
130+
- **Toasts and Modals**: Captures transient UI elements (success/error popups) separately.
131+
132+
### 2. Multi-Strategy Selector Healer (`selector-healer.ts`)
133+
Selectors in web apps frequently change. When a selector fails during execution, the healer runs 4 sequential recovery layers:
134+
135+
```
136+
[Target Selector fails]
137+
│
138+
▼
139+
1. Explicit Match ──► Locates element directly on active page (if successful -> Save in Memory)
140+
│ (failed)
141+
▼
142+
2. Memory Look-up ──► Checks previously healed selector history for this URL
143+
│ (failed)
144+
▼
145+
3. Role & Text ──► Finds element using ARIA role & text hints (e.g. button:has-text("Submit"))
146+
│ (failed)
147+
▼
148+
4. Indexed Match ──► Maps fallback matching using structural coordinates in browser state
149+
│ (failed)
150+
▼
151+
[Throw Selector Error / Mark Recoverable in Tool Loop]
152+
```
153+
154+
### 3. Two-Tier Safety Guard (`safety-guard.ts`)
155+
To prevent the agent from performing destructive actions in production/staging environments (such as bulk deletes, user invites, settings modifications, or real payments), the safety engine evaluates actions:
156+
- **Safe Tool Whitelist**: Tools that only observe or perform standard form interaction (e.g., `open_url`, `click_by_index`, `scroll`, `hover`) bypass filters immediately, preventing false positives.
157+
- **Intent Pattern Matching**: Unknown or custom tools are analyzed against safety rules (regex check) for action flags before execution. This prevents data fields (like entering `email: "delete-me@gmail.com"`) from triggering message-send blockages.
158+
159+
### 4. Zero-Dependency OOXML Excel Builder (`excel.ts`)
160+
To remain lightweight and portable, the Excel report generator uses **no external libraries** like `exceljs` or `xlsx`. It compiles raw OpenXML files directly:
161+
- Writes structure files: `[Content_Types].xml`, `xl/styles.xml`, `xl/workbook.xml`, `xl/worksheets/sheet1.xml`, etc.
162+
- Serializes screenshots into PNG files under `xl/media/` and writes `drawing.xml` elements to position screenshots inside cells.
163+
- Standardizes styling: formats headers (purple background, bold white text), severity tiers (Red/Critical, Amber/High, Yellow/Medium, Blue/Low), and column widths.
164+
- Bundles them using a lightweight, pure Node.js CRC32-based ZIP compiler.
165+
166+
### 5. Autonomous Explorer (`autonomous-explorer.ts`)
167+
In Codex/no-API mode, the agent isn't passive. It crawls and checks sites dynamically:
168+
- Locates navbar, sidebar, and tab navigation links.
169+
- Explores linked pages (restricted to the same origin URL).
170+
- Auto-detects forms, tests empty submit states, and runs validation auditors.
171+
- Takes screenshots of each path and merges issues (deduplicated by title) into the report.
172+
173+
---
174+
175+
## ⚡ Execution Lifecycle
176+
177+
For every QA test execution:
178+
179+
```
180+
1. Parse Arguments ──► 2. Initialize Playwright Context & Listeners
181+
│
182+
▼
183+
3. Execute Smart Login (if configured)
184+
│
185+
▼
186+
4. Execute Explicit Task Steps
187+
│
188+
▼
189+
5. Run Autonomous Explorer (crawls & fills)
190+
│
191+
▼
192+
6. Run QA Detectors & Playbooks
193+
│
194+
▼
195+
7. Compile Coverage Summary
196+
│
197+
▼
198+
8. Compile Excel Workbook + Media Zip
199+
│
200+
▼
201+
9. Update Site History & Memory
202+
```
203+
204+
---
205+
206+
## 🚀 Future Roadmap & Optimizations
207+
208+
1. **Local LLM-Driven Selector Healing**: Integrate a fallback to query a local model to match a modified DOM node when structural heuristics fail.
209+
2. **Visual Regression / Pixelmatch**: Capture visual baselines and diff screenshot outputs to highlight layout anomalies.
210+
3. **Structured Logger**: Introduce a unified, JSON-formatted structured logging engine (like `pino`) for improved monitoring.
211+
4. **Interactive Dashboard**: Build an HTML reporter utilizing charts to summarize historical run trends across test executions.

0 commit comments

Comments
 (0)