Skip to content

Commit 7532d96

Browse files
committed
docs: modularize GEMINI.md into Gemini CLI Skills
This commit transitions the large, monolithic project context in `GEMINI.md` into modular Gemini CLI Skills stored in the `skills/` directory. Benefits: - Reduced token overhead by leveraging Progressive Disclosure (triggering specific skills only when relevant). - Improved agent reliability with more focused, domain-specific instructions. - Simplified maintenance of specialized architectural and workflow documentation. Key Changes: - Created 7 new skills: backend, frontend, e2e, ingestion, workers, search-grammar, and maintenance. - Added `make link-skills` to the `Makefile` for easy skill registration. - Automated skill linking in `.devcontainer/post_attach.sh`. - Trimmed `GEMINI.md` to a high-level entry point that points to the skills.
1 parent afed110 commit 7532d96

File tree

19 files changed

+699
-514
lines changed

19 files changed

+699
-514
lines changed

.devcontainer/post_attach.sh

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,9 @@ else
5454
echo "You may need to configure an API key manually to use certain Gemini features."
5555
fi
5656

57+
# Link Gemini Skills
58+
echo "Linking Gemini Skills..."
59+
make link-skills
5760

5861
echo "Note: After starting the Gemini CLI for the first time, it may present you with some errors."
5962
echo "In that case, it will tell you to open a new terminal and start the Gemini CLI again."

GEMINI.md

Lines changed: 19 additions & 514 deletions
Large diffs are not rendered by default.

Makefile

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -553,3 +553,15 @@ download-playwright-report-from-run-%: check-gh-login
553553
rm -rf playwright-report
554554
mkdir -p playwright-report
555555
gh run download -R $(GH_REPO) $* -n playwright-report --dir playwright-report
556+
557+
################################
558+
# Skills
559+
################################
560+
link-skills:
561+
gemini skills link skills/webstatus-workers --consent
562+
gemini skills link skills/webstatus-backend --consent
563+
gemini skills link skills/webstatus-frontend --consent
564+
gemini skills link skills/webstatus-e2e --consent
565+
gemini skills link skills/webstatus-ingestion --consent
566+
gemini skills link skills/webstatus-search-grammar --consent
567+
gemini skills link skills/webstatus-maintenance --consent

skills/webstatus-backend/SKILL.md

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
---
2+
name: webstatus-backend
3+
description: Use when creating or modifying Go backend API endpoints, modifying Spanner database schemas, or working with OpenAPI and Spanner mappers.
4+
---
5+
6+
# webstatus-backend
7+
8+
This skill provides guidance for developing the Go-based backend API for `webstatus.dev`.
9+
10+
## Core Components
11+
12+
- **HTTP Server (`backend/pkg/httpserver`)**: Handles routing and requests via `oapi-codegen` stubs.
13+
- **Storage Adapter (`lib/gcpspanner/spanneradapters`)**: Translates API types to database types.
14+
- **Spanner Client (`lib/gcpspanner`)**: Core logic for Spanner interactions using the Mapper pattern.
15+
16+
## Guides
17+
18+
- **[Add a New API Endpoint](references/add-api-endpoint.md)**: Mandatory spec-first process.
19+
- **[Spanner Mapper Pattern](references/mapper-pattern.md)**: How to use the generic entity helpers.
20+
- **[Spanner Best Practices](references/spanner-best-practices.md)**: Efficient and safe querying.
21+
- **[Shared Libraries & Utilities](references/shared-libraries.md)**: Guidelines for `lib/`, `util/`, and the `OptionallySet` pattern.
22+
23+
## General Do's and Don'ts
24+
25+
- **DO** use `spanneradapters` for DB interactions in the API.
26+
- **DON'T** call `gcpspanner.Client` directly from `httpserver` handlers.
27+
- **DO** use `row.ToStruct(&yourStruct)` instead of manual column scanning.
28+
- **DO** define new Spanner table structs and query logic within `lib/gcpspanner`.
29+
- **DO** write integration tests using `testcontainers-go` for any changes to the `lib/gcpspanner` layer.
30+
- **DO** add response caching for new read-only endpoints in `backend/pkg/httpserver/cache.go`.
31+
- **DON'T** import `lib/backendtypes` into `lib/gcpspanner` (prevents circular dependencies).
32+
- **DO** handle business key to internal ID translation inside the `gcpspanner` client.
33+
- **DO** ensure `Merge` functions in mappers copy ALL fields, including `UpdatedAt`.
34+
- **DO** use `...WithTransaction` variants of helpers when inside a `ReadWriteTransaction`.
35+
36+
## Testing & Linting
37+
38+
- **Precommit Suite**: Run `make precommit` to execute the full suite of Go tests, formatting, and linting.
39+
- **Linting**: Run `make go-lint` to lint all Go code using `golangci-lint`.
40+
- **Quick Test Iteration**: Because this project uses a multi-module workspace (`go.work`), to run tests quickly for a single package without running the whole suite, execute `go test` from _within_ the specific module directory, or provide the full module path:
41+
```bash
42+
cd backend && go test -v ./pkg/...
43+
# Or
44+
go test -v github.com/GoogleChrome/webstatus.dev/lib/gcpspanner/...
45+
```
46+
- **Integration Tests**: Any changes to `lib/gcpspanner` **MUST** include integration tests using `testcontainers-go` against the Spanner emulator.
47+
48+
## Documentation Updates
49+
50+
When making significant architectural changes, adding new major endpoints, or altering the database schema:
51+
52+
- Trigger the "Updating the Knowledge Base" prompt in `GEMINI.md` to ensure I am aware of the changes.
53+
- Update `docs/ARCHITECTURE.md` if the system boundaries change.
54+
- Update these very skills files if you introduce new established patterns.
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# How to Add a New Backend API Endpoint
2+
3+
This project follows a **mandatory "spec-first"** process. You must define the API contract before writing any Go code to avoid compilation errors from missing types.
4+
5+
## Step-by-Step Process
6+
7+
### 1. Define the Contract (`openapi/backend/openapi.yaml`)
8+
9+
- Open `openapi/backend/openapi.yaml`.
10+
- Define the new endpoint, parameters, and request/response schemas.
11+
- If using new data structures, define them in `components.schemas`.
12+
- Ensure you set a clear `operationId`.
13+
14+
### 2. Generate Types
15+
16+
- Run `make openapi`.
17+
- This reads the YAML and generates Go server stubs, request/response structs, and TypeScript client code in `lib/gen/backend/`.
18+
- **CRITICAL:** Do not proceed until this command completes successfully.
19+
20+
### 3. Implement the HTTP Handler (`backend/pkg/httpserver/`)
21+
22+
- Add a new method to the `Server` struct. The method name must match the `operationId`.
23+
- This handler parses the request, calls the adapter layer, and writes the response.
24+
25+
### 4. Implement the Adapter Method (`lib/gcpspanner/spanneradapters/`)
26+
27+
- Add a method to the `Backend` struct.
28+
- Translate generated API types (from `lib/gen/backend/`) to internal Spanner types.
29+
- Call the underlying Spanner client.
30+
31+
### 5. Implement the Spanner Client Method (`lib/gcpspanner/`)
32+
33+
- Add a method to the `Client` struct.
34+
- Contains the actual database logic (queries, writes, transactions), ideally using the mapper pattern.
35+
36+
### 6. Add Tests
37+
38+
- **Unit Test**: In `lib/gcpspanner/spanneradapters/backend_test.go` using mocks.
39+
- **Integration Test**: In `lib/gcpspanner/*_test.go` using `testcontainers-go` against a Spanner emulator.
Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
# The Go Mapper Pattern for Spanner
2+
3+
Used for all interactions with the Spanner database to reduce boilerplate and ensure consistency. Defined in `lib/gcpspanner/client.go`.
4+
5+
## Core Concept
6+
7+
Use generic helpers configured with a "mapper" struct instead of custom query logic.
8+
9+
### Generic Helpers
10+
11+
- `newEntityReader`
12+
- `newEntityWriter`
13+
- `newEntitySynchronizer`
14+
15+
### Mapper Interfaces
16+
17+
- `baseMapper`: Defines the Spanner table name.
18+
- `readOneMapper`: Defines selection by key.
19+
- `mergeMapper`: Defines how to merge updates.
20+
- `deleteByStructMapper`: Defines deletion.
21+
- `childDeleteMapper`: Handles child deletions in batches (via `GetChildDeleteKeyMutations`) to stay under Spanner's mutation limit.
22+
23+
## Transactional Usage
24+
25+
- **DO** use the `...WithTransaction` variants (e.g., `createWithTransaction`, `updateWithTransaction`) inside a `ReadWriteTransaction`.
26+
- **DON'T** use standard helpers inside a transaction; they will attempt to start a new transaction and fail.
27+
- **DON'T** use `spanner.InsertStruct` manually.
28+
29+
## Implementation Guardrails
30+
31+
- **Merge Logic**: Ensure `Merge` or `mergeAndCheckChangedMapper` copies **all** fields. A missing field assignment will cause silent update failures.
32+
- **UpdatedAt**: Always set `UpdatedAt` to `spanner.CommitTimestamp` in the input struct before merging.
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Shared Go Libraries & Utilities
2+
3+
The `lib/` directory contains shared code used by the `backend` and `workflows`.
4+
5+
## Key Subdirectories
6+
7+
- **`lib/gen`**: Auto-generated code (OpenAPI, JSON Schema, ANTLR). **Never edit manually.**
8+
- **`lib/gcpspanner`**: Spanner client, models, generic mapper helpers.
9+
- **`lib/gcpspanner/spanneradapters`**: Translation layer between external services and the DB client.
10+
- **`lib/generic`**: Utilities like `OptionallySet[T]` for handling optional fields.
11+
- **`lib/blobtypes`**: GCS blob storage definitions.
12+
13+
## Blob Storage & Schema Evolution
14+
15+
When storing persistent state (like saved search notifications in GCS blobs):
16+
17+
- **Canonical vs. Storage Types**: Keep internal business logic (`lib/workertypes/comparables`) separate from persistent types (`lib/blobtypes/v1`).
18+
- **The `OptionallySet` Pattern**: To handle forward and backward compatibility, wrap new struct fields in `generic.OptionallySet[T]`.
19+
- _Old Blob_: Field missing -> `IsSet: false`. Logic ignores it (Quiet Rollout).
20+
- _New Data_: Field present -> `IsSet: true`. Logic processes it.
21+
- **DON'T** change the meaning of existing fields in `lib/blobtypes`; create `v2` if a breaking change is needed.
22+
23+
## Utility Scripts
24+
25+
Small CLI tools and helper scripts reside in `util/`:
26+
27+
- `util/run_job.sh`: Runs data ingestion locally via Minikube.
28+
- `util/cmd/load_fake_data/`: Emulators fake data (`make dev_fake_data`).
29+
- `util/cmd/load_test_users/`: Emulators fake users (`make dev_fake_users`).
30+
- **DON'T** put production application logic in `util/`.
31+
32+
## Verify, Don't Assume
33+
34+
Always consult the canonical sources of truth instead of assuming based on general patterns:
35+
36+
- **API Contracts**: `openapi/backend/openapi.yaml`
37+
- **Database Schema**: `infra/storage/spanner/migrations/`
38+
- **Search Grammar**: `antlr/FeatureSearch.g4`
39+
- **External Schemas**: `jsonschema/`
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Go Spanner Query Best Practices
2+
3+
## Database Migrations
4+
5+
- **Creation**: Run `make spanner_new_migration` to create a new migration file in `infra/storage/spanner/migrations/`.
6+
- **Data Migrations**: For complex data migrations (e.g. renaming a feature key), use the generic migrator in `lib/gcpspanner/spanneradapters/migration.go`.
7+
8+
## Scanning Rows
9+
10+
- **DO** use `row.ToStruct(&yourStruct)` to scan results. It leverages struct tags (e.g., `spanner:"ColumnName"`) and is less error-prone.
11+
- **DON'T** manually scan columns using `r.ColumnByName`. This is verbose and fragile to schema changes.
12+
13+
## Foreign Keys
14+
15+
- **Cascade Deletes**: Use `ON DELETE CASCADE` for relationships.
16+
- **Batched Deletes**: If a cascade would delete thousands of rows, implement `GetChildDeleteKeyMutations` in the parent mapper to delete children in batches first.
17+
18+
## Mappers vs Clients
19+
20+
- **DO** look for existing mappers in `lib/gcpspanner/` (e.g. `webFeatureSpannerMapper`) before creating new ones.
21+
- **DO** translate business keys to internal IDs inside the `gcpspanner` client so that adapters/workflows remain unaware of internal DB IDs.

skills/webstatus-e2e/SKILL.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
---
2+
name: webstatus-e2e
3+
description: Use when writing, modifying, or debugging Playwright end-to-end (E2E) tests for webstatus.dev.
4+
---
5+
6+
# webstatus-e2e
7+
8+
This skill provides guidance for working with the End-to-End (E2E) test suite in `webstatus.dev`, which is built using Playwright and TypeScript.
9+
10+
## Architecture & Location
11+
12+
- **Directory**: The E2E tests are located in the `e2e/` directory.
13+
- **Framework**: **Playwright** with **TypeScript**.
14+
- **Configuration**: `playwright.config.ts` handles browser definitions, retries, and worker limits.
15+
16+
## Guidelines (Do's and Don'ts)
17+
18+
- **DO** add E2E tests for critical user journeys (e.g., login flows, complex search operations, saving a search).
19+
- **DON'T** write E2E tests for small component-level interactions; those belong in frontend unit tests (`frontend/src/**/*.test.ts`).
20+
- **DO** use resilient locators. Prefer using `data-testid` attributes (e.g., `page.getByTestId('submit-btn')`) over brittle CSS classes or XPath.
21+
- **DO** move the mouse to a neutral position (e.g., `page.mouse.move(0, 0)`) before taking visual snapshots to avoid flaky tests caused by unintended hover effects on UI elements.
22+
23+
## Configuration & Stability
24+
25+
- **Single Worker**: Tests currently operate on the same end-user accounts, which means they can interfere with each other if run concurrently. To ensure stability, `workers: 1` is strictly enforced in `playwright.config.ts`.
26+
- **Retries**: Playwright tests are configured to retry twice on failure only when running in a CI environment. If you want to simulate this locally and test flakiness, you can prefix your command with `CI=true` (e.g., `CI=true make playwright-test`).
27+
- **Browsers**: If you ever need to test against new browsers (e.g., mobile viewports, branded Edge/Chrome), modify the `projects` array within `playwright.config.ts`.
28+
29+
## Execution & Debugging
30+
31+
- For detailed instructions on rapid iteration, debugging CI failures, and using traces, see [references/execution-and-debugging.md](references/execution-and-debugging.md).
32+
33+
## Commands Summary
34+
35+
- Use the `Makefile` in the project root:
36+
- `make playwright-test`: Sets up a fresh local environment and runs the test suite.
37+
- `make playwright-ui`: Runs the tests in Playwright's interactive UI mode.
38+
- `make playwright-debug`: Runs the tests in debug mode.
39+
- `make playwright-update-snapshots`: Updates visual regression snapshots.
40+
41+
## Documentation Updates
42+
43+
When modifying playwright configuration, retries, or execution strategies:
44+
45+
- Trigger the "Updating the Knowledge Base" prompt in `GEMINI.md` to ensure I am aware of the changes.
46+
- Update these very skills files if you introduce new established patterns for testing.
Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Playwright Execution and Debugging
2+
3+
## Rapid Iteration (`SKIP_FRESH_ENV`)
4+
5+
If you already have a local environment running (via `make start-local`, `make port-forward-manual`, `make dev_fake_users`, and `make dev_fake_data`), you can skip the environment teardown and setup to run tests faster.
6+
7+
**Command**: `SKIP_FRESH_ENV=1 make playwright-test`
8+
9+
_Note: Use this only when your local environment is in a known good state._
10+
11+
## Debugging GitHub Failures
12+
13+
When E2E tests fail in CI, you can download the traces and reports locally for analysis.
14+
15+
1. **Find the Run ID**: `make print-gh-runs`
16+
2. **Download the Report**: `make download-playwright-report-from-run-<RUN_ID>`
17+
3. **Open the Report**: `make playwright-open-report`
18+
4. **Inspect Traces**:
19+
- List traces: `make playwright-show-traces`
20+
- Open a specific trace: `npx playwright show-trace playwright-report/data/<trace_hash>.zip`
21+
22+
## Local Interactive Tools
23+
24+
- **Playwright UI**: `make playwright-ui` (best for seeing tests run live and inspecting locators).
25+
- **Step-through Debugging**: `make playwright-debug`.

0 commit comments

Comments
 (0)