ci: add automated skill review for SKILL.md pull requests#6650
ci: add automated skill review for SKILL.md pull requests#6650popey wants to merge 1 commit intoelizaOS:developfrom
Conversation
Adds a lightweight GitHub Action that reviews any SKILL.md files changed in a pull request using tessl skill review. Posts results as a PR comment with minimal permissions (pull-requests: write, contents: read).
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| pull_request: | ||
| paths: ['**/SKILL.md'] |
There was a problem hiding this comment.
The pull_request event runs in a restricted token context for fork PRs — even though pull-requests: write is specified, GitHub limits the GITHUB_TOKEN to read-only for fork-originated PRs. Since most external contributors will open PRs from forks, the tesslio/skill-review action will fail to post a comment without any visible error to the author.
The common fix is to use pull_request_target instead. However, pull_request_target runs in the context of the base branch (with full secrets access), so it requires careful handling to avoid running untrusted code. A safe pattern is to combine pull_request_target with the explicit checkout of the PR head:
on:
pull_request_target:
paths: ['**/SKILL.md']Note: be aware that pull_request_target gives the action access to repository secrets and write permissions even for fork PRs, so this is a trade-off to discuss with the team before adopting.
| contents: read | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: tesslio/skill-review@940d4a129aa5bce76f007078e2975c4ea3c74976 # main |
There was a problem hiding this comment.
tesslio/skill-review is from an external organization with minimal public adoption (1 star, 0 forks at the time of this review). While pinning to a commit SHA (940d4a129aa5bce76f007078e2975c4ea3c74976) is good practice, this action is granted contents: read (full source access after actions/checkout) and pull-requests: write (can post comments on behalf of the repo).
Consider auditing the action's source code at that commit before merging, to confirm it only reads SKILL.md and posts a comment — and does not exfiltrate code or secrets to an external endpoint.
| jobs: | ||
| review: | ||
| runs-on: ubuntu-latest |
There was a problem hiding this comment.
concurrency group and timeout-minutes
Other workflows in this repo (e.g., claude-code-review.yml) define a concurrency group to cancel superseded runs when a PR is force-pushed or updated rapidly. Without it, multiple runs of skill-review can queue up for the same PR. A timeout-minutes limit also prevents a hung action from consuming minutes indefinitely.
| jobs: | |
| review: | |
| runs-on: ubuntu-latest | |
| jobs: | |
| review: | |
| runs-on: ubuntu-latest | |
| timeout-minutes: 10 | |
| concurrency: | |
| group: skill-review-${{ github.event.pull_request.number }} | |
| cancel-in-progress: true |
1 participant
Hullo! Thanks for merging the skill improvements earlier. This is a follow-up that adds a lightweight GitHub Action to automatically review any
SKILL.mdfiles when they're changed in a PR, using tessl skill review.**/SKILL.mdpull-requests: writeandcontents: readThis way you and your contributors get an instant quality signal on skill changes before manual review — no signup or tokens needed.