Skip to content

Commit 3ea8f30

Browse files
committed
docs: polish watchlist skill guardrails
1 parent 2988e37 commit 3ea8f30

6 files changed

Lines changed: 99 additions & 16 deletions

File tree

.agents/skills/watchlist-md/SKILL.md

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,11 @@ name: watchlist-md
33
description: >-
44
Manages WATCHLIST.md entries for explicit user-requested deferred checks and
55
lifecycle updates. Use when the user mentions WATCHLIST.md, a WL-YYYYMMDD-NNN
6-
item ID, WATCHLIST.md에 추가, watchlist로 남겨, 후속 체크로 기록, pending
7-
result, or asks to record time/event-gated CI, deploy, job, data sync, order,
8-
ticket, PR, or email follow-up. Do not trigger for generic reminder/scheduler
9-
requests unless the user explicitly asks to record a WATCHLIST.md note.
6+
item ID, WATCHLIST.md에 추가, watchlist로 남겨, 후속 체크로 기록, or asks to
7+
record a pending result for later review, time/event-gated CI, deploy, job,
8+
data sync, order, ticket, PR, or email follow-up. Do not trigger for generic
9+
reminder/scheduler requests unless the user explicitly asks to record a
10+
WATCHLIST.md note.
1011
Lifecycle words such as 완료, 삭제, 취소, 드롭, 차단, 연기, 보관, and 아카이브
1112
apply only when they clearly refer to WATCHLIST.md or a WL-YYYYMMDD-NNN item.
1213
Records notes only; never schedules reminders or wakeups without an explicitly
@@ -48,6 +49,10 @@ Add an item only when the user explicitly asks to record a future, time-gated, o
4849
event-gated check, or has opted into pre-authorized watchlist recording. If the
4950
task can reasonably be completed now, do that instead.
5051

52+
Pre-authorized watchlist recording is scoped to the current repository/workspace
53+
and active workflow unless the user explicitly says otherwise. Do not treat it as
54+
global memory.
55+
5156
Use this item shape:
5257

5358
```md
@@ -57,7 +62,7 @@ Use this item shape:
5762
- owner: user|assistant_on_review|both|external
5863
- due_at: YYYY-MM-DDTHH:MM:SS+09:00
5964
- created_at: YYYY-MM-DDTHH:MM:SS+09:00
60-
- source: short stable pointer, link, file, PR, issue, or conversation note
65+
- source: short stable pointer, safe link, file, PR, issue, or conversation note
6166
- trigger: why this needs a later check
6267
- action: what to check or do
6368
- done_when: observable success condition
@@ -66,22 +71,27 @@ Use this item shape:
6671
- next_step_on_fail:
6772
```
6873

69-
Required information: ID, status, due time, owner, action, done condition, and
70-
source/context. Use `assistant_on_review` when the assistant should help on the
71-
next explicit review. Treat legacy `owner: agent` as `assistant_on_review`.
74+
Required information for open items: ID, status, priority, owner, due_at value,
75+
created_at, source, trigger, action, and done_when. Recommended when known:
76+
next_step_on_fail. Normally blank until checked: last_checked_at and result. Do
77+
not store signed, tokenized, private, or credential-bearing links. Use
78+
`assistant_on_review` when the assistant should help on the next explicit review.
79+
Treat legacy `owner: agent` as `assistant_on_review`.
7280

73-
Generate IDs as `WL-YYYYMMDD-NNN` from the creation date in Asia/Seoul unless the
74-
file or user specifies another timezone. Before writing, re-read WATCHLIST.md and
75-
choose the next unused sequence for that date. Never overwrite an existing item.
81+
Generate IDs as `WL-YYYYMMDD-NNN` from the creation date in the WATCHLIST
82+
timezone. Prefer the WATCHLIST.md `timezone:` field, then explicit user timezone,
83+
then environment/user timezone; if none is available, use Asia/Seoul. Before
84+
writing, re-read WATCHLIST.md and choose the next unused sequence for that date.
85+
Never overwrite an existing item.
7686

7787
Convert relative times to absolute ISO-8601 timestamps with timezone when possible.
7888
If time is ambiguous, use `due_at: unscheduled` and mention the ambiguity. If the
7989
resolved time is already in the past, ask whether to record it or use the next
8090
occurrence; if clarification is unavailable, use `unscheduled`.
8191

82-
After adding, confirm the item ID, due time, action, done condition, and scheduler
83-
status. If no scheduler was used, say `scheduler: none` and avoid promising future
84-
execution.
92+
After adding, confirm the item ID, due_at value, action, done_when, and scheduler
93+
status. If no scheduler was used, say `scheduler: none` and avoid promising
94+
future execution.
8595

8696
## Review
8797

.agents/skills/watchlist-md/references/lifecycle.md

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,9 @@ Git history cleanup separately; do not rewrite history unless explicitly asked.
7676

7777
## ID And Time Rules
7878

79-
- Generate IDs as `WL-YYYYMMDD-NNN` from the creation date in Asia/Seoul by
80-
default.
79+
- Generate IDs as `WL-YYYYMMDD-NNN` from the creation date in the WATCHLIST
80+
timezone. Prefer the WATCHLIST.md `timezone:` field, then explicit user
81+
timezone, then environment/user timezone; if none is available, use Asia/Seoul.
8182
- Use the next `NNN` for that date by reading existing item IDs.
8283
- Immediately before writing, re-read WATCHLIST.md and scan all existing IDs. If
8384
the chosen ID already exists, increment `NNN` until an unused ID is found.

README.ko.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,8 @@ python3 evals/check_semantic_cases.py
200200

201201
`evals/prompts.csv`, `evals/rubric.md`, `evals/self_checks.yaml`, `evals/cases/*.json`은 수동 또는 자동 에이전트 평가에 사용할 작은 프롬프트 회귀 세트입니다. Semantic case checker는 기대 trigger와 operation 계약을 검증하며, LLM 또는 agent를 실행하지 않습니다.
202202

203+
`--strict-safety`는 의도적으로 보수적입니다. 공유/팀 템플릿에서는 signed URL 또는 tokenized URL처럼 보이는 휴리스틱 결과도 error로 올립니다. false positive는 검토하고, 민감한 링크를 WATCHLIST.md에 복사하기보다 safe pointer를 선호하세요.
204+
203205
## Example Item
204206

205207
```md
@@ -220,6 +222,8 @@ python3 evals/check_semantic_cases.py
220222

221223
`owner`는 다음 명시적인 WATCHLIST 검토 중에 누가 조치해야 하는지를 의미합니다. 이는 어시스턴트가 자동으로 깨어난다는 의미는 아닙니다.
222224

225+
검증기는 모든 필드 키를 요구합니다. 키는 위의 안정적인 순서대로 있어야 하지만, open 항목에서 모든 필드 값이 채워져야 하는 것은 아닙니다. open 항목의 필수 값은 `status`, `priority`, `owner`, `due_at`, `created_at`, `source`, `trigger`, `action`, `done_when`입니다. 알 수 있으면 권장되는 값은 `next_step_on_fail`입니다. 확인 전에는 보통 비워 둡니다: `last_checked_at`, `result`.
226+
223227
완료 처리의 기본 동작은 `status: done`, `last_checked_at`, `result`를 채우고, `## Done` 섹션이 있으면 완료 항목을 그 아래로 이동하는 것입니다. 사용자가 “상태만 바꿔” 또는 “위치 유지”처럼 명시하면 항목을 원래 위치에 둘 수 있습니다.
224228

225229
`dropped`는 더 이상 필요 없는 후속 확인의 기록을 보존하는 상태입니다. Delete는 기록 자체를 제거하는 동작이므로 기본적으로 권장하지 않으며, 사용자가 명시적으로 삭제를 요청할 때만 수행해야 합니다.

README.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -200,6 +200,8 @@ python3 evals/check_semantic_cases.py
200200

201201
`evals/prompts.csv`, `evals/rubric.md`, `evals/self_checks.yaml`, and `evals/cases/*.json` are a small prompt regression set for manual or automated agent evaluation. The semantic case checker validates the expected trigger and operation contract; it does not run an LLM or agent.
202202

203+
`--strict-safety` is intentionally conservative. It escalates heuristic findings such as signed or tokenized-looking URLs to errors for shared/team templates; review false positives and prefer safe pointers instead of copying sensitive links into WATCHLIST.md.
204+
203205
## Example Item
204206

205207
```md
@@ -220,6 +222,8 @@ python3 evals/check_semantic_cases.py
220222

221223
`owner` means who should act during the next explicit WATCHLIST review. It does not mean the assistant will wake up automatically.
222224

225+
The validator requires every field key in the stable order shown above, but not every field needs a populated value for an open item. Required values for open items are `status`, `priority`, `owner`, `due_at`, `created_at`, `source`, `trigger`, `action`, and `done_when`. Recommended when known: `next_step_on_fail`. Normally blank until checked: `last_checked_at` and `result`.
226+
223227
By default, completing an item sets `status: done`, fills `last_checked_at` and `result`, and moves the item under `## Done` when that section exists. If the user explicitly says to change only the status or keep the item in place, leave the item in its original section.
224228

225229
`dropped` preserves a record for a follow-up that is no longer needed. Delete removes the record itself, so it is not the default and should only be used when the user explicitly asks to delete the record.

evals/check_policy_markers.py

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,8 @@
1414
"Concurrent Edits",
1515
"Do not store passwords, tokens",
1616
"untrusted",
17+
"`--strict-safety` is intentionally conservative",
18+
"Required values for open items",
1719
],
1820
"README.ko.md": [
1921
"Deletion And Retention Policy",
@@ -23,6 +25,8 @@
2325
"Concurrent Edits",
2426
"비밀번호, 토큰",
2527
"신뢰할 수 없는",
28+
"`--strict-safety`는 의도적으로 보수적입니다",
29+
"open 항목의 필수 값",
2630
],
2731
".agents/skills/watchlist-md/SKILL.md": [
2832
"Lifecycle words such as",
@@ -32,13 +36,22 @@
3236
"untrusted data",
3337
"references/lifecycle.md",
3438
"references/safety.md",
39+
"pending result for later review",
40+
"safe link",
41+
"Pre-authorized watchlist recording is scoped",
42+
"Required information for open items",
43+
"due_at value",
44+
"WATCHLIST.md `timezone:` field",
45+
"environment/user timezone",
3546
],
3647
".agents/skills/watchlist-md/references/lifecycle.md": [
3748
"Deletion And Retention Policy",
3849
"Do not archive items automatically",
3950
"Archive Policy",
4051
"Concurrent Edit And ID Collision Policy",
4152
"List-only reviews must not mutate",
53+
"WATCHLIST.md `timezone:` field",
54+
"environment/user timezone",
4255
],
4356
".agents/skills/watchlist-md/references/safety.md": [
4457
"Do not store secrets",

evals/test_check_watchlist.py

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -581,6 +581,57 @@ def test_skill_runtime_guidance_stays_lean(self):
581581
self.assertIn("references/safety.md", text)
582582
self.assertLess(text.index("## Add"), text.index("references/lifecycle.md"))
583583

584+
def test_skill_runtime_polish_markers_stay_precise(self):
585+
text = (SKILL_DIR / "SKILL.md").read_text(encoding="utf-8")
586+
lifecycle = (SKILL_DIR / "references" / "lifecycle.md").read_text(encoding="utf-8")
587+
normalized_text = " ".join(text.split()).lower()
588+
normalized_lifecycle = " ".join(lifecycle.split())
589+
timezone_precedence = (
590+
"Prefer the WATCHLIST.md `timezone:` field, then explicit user timezone, "
591+
"then environment/user timezone; if none is available, use Asia/Seoul."
592+
)
593+
required_information = (
594+
"Required information for open items: ID, status, priority, owner, "
595+
"due_at value, created_at, source, trigger, action, and done_when."
596+
)
597+
598+
self.assertIn("pending result for later review", text)
599+
self.assertNotIn("후속 체크로 기록, pending", text)
600+
self.assertIn(
601+
"pre-authorized watchlist recording is scoped to the current repository/workspace",
602+
normalized_text,
603+
)
604+
self.assertIn(
605+
"- source: short stable pointer, safe link, file, PR, issue, or conversation note",
606+
text,
607+
)
608+
self.assertIn("due_at value", text)
609+
self.assertNotIn("due time", text)
610+
self.assertIn(required_information, " ".join(text.split()))
611+
self.assertNotIn("done condition", text)
612+
self.assertIn("action, done_when, and scheduler status", " ".join(text.split()))
613+
self.assertIn("watchlist timezone", normalized_text)
614+
self.assertIn("environment/user timezone", normalized_text)
615+
self.assertIn(timezone_precedence, " ".join(text.split()))
616+
self.assertIn(timezone_precedence, normalized_lifecycle)
617+
618+
def test_readme_documents_field_and_strict_safety_expectations(self):
619+
english = (REPO_ROOT / "README.md").read_text(encoding="utf-8")
620+
korean = (REPO_ROOT / "README.ko.md").read_text(encoding="utf-8")
621+
622+
self.assertIn("The validator requires every field key", english)
623+
self.assertIn("Required values for open items", english)
624+
self.assertIn("`source`, `trigger`, `action`, and `done_when`", english)
625+
self.assertIn("Recommended when known", english)
626+
self.assertIn("Normally blank until checked", english)
627+
self.assertIn("`--strict-safety` is intentionally conservative", english)
628+
self.assertIn("검증기는 모든 필드 키를 요구합니다", korean)
629+
self.assertIn("open 항목의 필수 값", korean)
630+
self.assertIn("`source`, `trigger`, `action`, `done_when`", korean)
631+
self.assertIn("알 수 있으면 권장되는 값", korean)
632+
self.assertIn("확인 전에는 보통 비워 둡니다", korean)
633+
self.assertIn("`--strict-safety`는 의도적으로 보수적입니다", korean)
634+
584635
def test_starter_templates_label_commented_item_as_example_only(self):
585636
paths = [
586637
SKILL_DIR / "assets" / "WATCHLIST.template.md",

0 commit comments

Comments
 (0)