Audit candidate: caliber-ai-org/ai-setup

[xiaolai]

Repo

https://github.com/caliber-ai-org/ai-setup

Field	Value
Stars	698
Star velocity	5.7/day
Star anomaly	1.3× baseline
NL Artifacts	24
Last updated	0 days ago
Source	claudepot daily feed

Next Steps

1. Batch processor picks this up on next cycle → labels audit-ready
2. Audit workflow dispatches automatically (via fix in 81fbc7d)
3. Audit runs security scan + NL scoring

Checklist

• Accepts contributions
• Updated in last 60 days (confirmed: 0 days)
• Not a fork or mirror
• Has meaningful NL artifacts (confirmed: 24)

[github-actions]

Audit Complete: caliber-ai-org/ai-setup

NL Score: 100/100 | Security: CLEAR | Artifacts: 25 | Strategy: batched

NLPM Audit: caliber-ai-org/ai-setup

Date: 2026-04-20 | Artifacts: 25 | Strategy: batched
NL Score: 100/100
Security: CLEAR
Bugs: 4 | Quality Issues: 4 | Security Findings: 4

NL Score Summary

File	Type	Score	Top Issue
.claude/skills/scoring-checks/SKILL.md	skill	98	Vague: "appropriate category section"
.claude/skills/save-learning/SKILL.md	skill	98	Vague: "appropriate type prefix"
.agents/skills/save-learning/SKILL.md	skill	98	Vague: "appropriate type prefix"
.cursor/skills/save-learning/SKILL.md	skill	98	Vague: "appropriate type prefix"
.claude/skills/llm-provider/SKILL.md	skill	100	None
.claude/skills/caliber-testing/SKILL.md	skill	100	None
.claude/skills/adding-a-command/SKILL.md	skill	100	None
.claude/skills/writers-pattern/SKILL.md	skill	100	None
.claude/skills/setup-caliber/SKILL.md	skill	100	None
.claude/skills/find-skills/SKILL.md	skill	100	None
.agents/skills/llm-provider/SKILL.md	skill	100	None
.agents/skills/scoring-checks/SKILL.md	skill	100	None
.agents/skills/caliber-testing/SKILL.md	skill	100	None
.agents/skills/adding-a-command/SKILL.md	skill	100	None
.agents/skills/writers-pattern/SKILL.md	skill	100	None
.agents/skills/setup-caliber/SKILL.md	skill	100	None
.agents/skills/find-skills/SKILL.md	skill	100	None
.cursor/skills/llm-provider/SKILL.md	skill	100	None
.cursor/skills/scoring-checks/SKILL.md	skill	100	None
.cursor/skills/caliber-testing/SKILL.md	skill	100	None
.cursor/skills/adding-a-command/SKILL.md	skill	100	None
.cursor/skills/writers-pattern/SKILL.md	skill	100	None
.cursor/skills/setup-caliber/SKILL.md	skill	100	None
.cursor/skills/find-skills/SKILL.md	skill	100	None
CLAUDE.md	context	100	None

Weighted average: (21 × 100 + 4 × 98) / 25 = 99.68 → 100/100

Security Scan

Severity	Count
Critical	0
High	0
Medium	2
Low	2

Execution Surface Inventory

Surface	Files
Hooks	None
Scripts	scripts/postinstall.js, scripts/demo-init.sh, scripts/demo-output.sh, scripts/demo-shell.sh

---

Full report: https://github.com/xiaolai/nlpm-for-claude/blob/main/auditor/audits/caliber-ai-org-ai-setup.md

7 bug(s) found. To submit fix PRs to the target repo, add the contribute-approved label to this issue.

[xiaolai]

Contribution: caliber-ai-org/ai-setup

Audit score: 100/100 NL | Security: CLEAR (0 Critical/High, 2 Medium, 2 Low)

PRs submitted

PR	Title	Type
caliber-ai-org/ai-setup#178	fix(skills): align llm-provider stream() signature across platforms	Bug
caliber-ai-org/ai-setup#179	fix(skills): correct adding-a-command export pattern in .cursor/ skill	Bug
caliber-ai-org/ai-setup#180	fix(skills): correct writers-pattern sync/export signature across platforms	Bug
caliber-ai-org/ai-setup#181	fix(skills): correct scoring-checks function signature across platforms	Bug
caliber-ai-org/ai-setup#182	docs(telemetry): document PostHog analytics and opt-out in README	Security (Medium)

Tracking issue

caliber-ai-org/ai-setup#183

Root cause summary

All 4 bugs share the same root cause: platform skill copies (.agents/, .cursor/) that drifted out of sync with the authoritative .claude/ versions. The .claude/ versions correctly match the actual TypeScript source (src/llm/types.ts, src/writers/index.ts, src/scoring/checks/*.ts, src/cli.ts). The .agents/ and .cursor/ copies document non-existent interfaces that would produce TypeScript compile errors.

[github-actions]

All PRs resolved. 5 merged, 0 applied-separately, 0 rejected. Ready for case study.

[github-actions]

Case study published: case-studies/2026-04-25-caliber-ai-org-ai-setup.md — caliber-ai-org/ai-setup (737★): NL score 100/100, but cross-platform skill divergence produced 4 bugs with incompatible TypeScript signatures across .claude/.agents/.cursor — all 5 PRs (llm-provider, writers-pattern, adding-a-command, scoring-checks, telemetry disclosure) merged in 4 days; re-audit skipped.