[codex] Harden WATCHLIST validation - #7
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request upgrades the project to version 0.2.0, introducing a more robust validation system and expanded documentation. Key additions include strict safety scanning for secrets, format enforcement, and new CI scripts to verify policy markers and release metadata. Documentation was updated with a quickstart guide, security policies, and a PR template. A review comment correctly identified unreachable logic in the safety scanning function of the validator script, noting that the current implementation prevents the reporting of non-blocking warnings.
| if severity == "error" or strict_safety: | ||
| add_error(result, code, message, watch_id, field, severity=severity) | ||
| else: | ||
| add_warning(result, code, message, watch_id, field) |
There was a problem hiding this comment.
The else block (lines 282-283) is currently unreachable. The function scan_safety returns early if strict_safety is False (line 271). When strict_safety is True, the condition severity == "error" or strict_safety (line 280) is always true.
If safety scanning is intended to be a strictly opt-in feature that always blocks on findings, you can simplify this logic to always call add_error. If you intended to allow non-blocking safety warnings (e.g., for patterns marked as warning in SENSITIVE_PATTERNS) when not in strict mode, you should remove the early return at line 271 and adjust the condition at line 280.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c00f2e189e
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| fields["last_checked_at"], | ||
| allow_unscheduled=False, | ||
| ) | ||
| continue |
There was a problem hiding this comment.
When a WATCHLIST item is missing any required field, this continue skips scan_safety, so --strict-safety can fail only with MISSING_REQUIRED_FIELDS while never reporting an embedded token, signed URL, or raw private excerpt in the fields that were present. In that malformed-entry scenario, the new redaction/rotation guidance is withheld even though the unsafe value is still in the file; scan the parsed fields before continuing or otherwise report safety findings independently of format errors.
Useful? React with 👍 / 👎.
1 participant
Summary
0.2.0Validation
python3 -m unittest discover -s evals -p 'test_*.py'python3 evals/check_watchlist.py .watchlist/WATCHLIST.md --strict-safety --require-archive-sectionpython3 evals/check_watchlist.py .agents/skills/watchlist-md/assets/WATCHLIST.template.md --strict-safety --require-archive-sectionpython3 evals/check_release_metadata.pypython3 evals/check_policy_markers.pypython3 -m py_compile evals/check_watchlist.py evals/check_policy_markers.py evals/check_release_metadata.py evals/test_check_watchlist.pygit diff --check