Open-source tool scores AI agent SKILL.md files with a CI-friendly 0–100 rubric

I Pointed a Skill Linter at a 52k-Star Repo. Here Is What 84/100 Looks Like.

Every AI agent skill you write burns context on every turn. Not just when the skill is running. On every turn. The agent keeps each skill's name and description loaded permanently so it knows when to invoke them. A vague description is not just a documentation problem. It is a tax you pay per message, forever. That is the problem I built skillscore to catch. When addyosmani/agent-skills hit 52,000 stars and went to #1 trending on GitHub, I had my benchmark. 24 production-grade skills written by people who clearly know what they are doing. If a static linter has anything useful to say at this level, this is where to find out. So I ran it. One command. 24 skills. Two seconds. This is what skillscore 0.2.0 can do now: skillscore /path/to/agent-skills/ One command scores everything in the tree. Here is the output: Three skills from addyosmani/agent-skills scored in one command, then a drill-down into the lowest scorer. The full results Skill Score Grade spec-driven-development 91 A browser-testing-with-devtools 91 A deprecation-and-migration 91 A frontend-ui-engineering 91 A test-driven-development 88 B code-review-and-quality 88 B interview-me 86 B ci-cd-and-automation 85 B code-simplification 85 B context-engineering 85 B documentation-and-adrs 85 B incremental-implementation 85 B security-and-hardening 85 B shipping-and-launch 85 B source-driven-development 85 B using-agent-skills 85 B doubt-driven-development 80 B observability-and-instrumentation 80 B planning-and-task-breakdown 80 B api-and-interface-design 78 C debugging-and-error-recovery 77 C git-workflow-and-versioning 77 C idea-refine 77 C performance-optimization 77 C Average: 84/100 (B) To be clear: 84 across 24 production skills is excellent. No failures. No D grades. Most skill libraries I have tested do not get close to this. The instruction content inside these skills is genuinely good. What the linter found is at the edges, not in the core. Two gaps. Five skills. Every single C. I drilled into all five C-grade skills. The same two findings came up in every one of them. The description does not say when to stop Every C-grade description says what the skill does. None says when not to use it. This matters because an agent with no stop condition will stretch a skill to cover loosely related requests. It invokes when it should not. It does not know where the boundary is because you never told it. The fix is one sentence at the end of the description: "Do not use when the codebase already has an established pattern for this." That is it. One sentence. The skill immediately becomes less likely to activate on the wrong request. Terminal commands with no Safety section Several of the C-grade skills ship step-by-step terminal commands in the body. None of them has a ## Safety section. The Antigravity authoring guide requires any skill that runs commands to document what those commands touch, and what the agent must never run unattended. Without that section, the linter applies up to an 8-point penalty. The reason is practical: an agent executing undocumented commands has no signal about blast radius. Here is what a Safety section looks like: ## Safety - Never run `git push --force` unattended. Confirm with the user first. - All destructive commands require explicit confirmation before execution. - Scripts in scripts/ are reviewed before running, never piped directly to sh. Five lines. Eight points back. Add both of those things and every C-grade skill in this dataset moves to B or A territory. The instruction quality is already there. The metadata layer just needed these two signals. Running it yourself # Install dart pub global activate skillscore # Score your skills skillscore path/to/your-skills/ # Gate CI (fail if any skill drops below 80) skillscore skills/ --min-score 80 --format sarif --format sarif pipes findings into GitHub code scanning so they appear as inline annotations on pull requests. No more "I forgot to check the skill before merging." If a finding is unclear, skillscore explain <rule-id> prints the full rationale and the guide it came from. Every output line includes the rule ID for exactly this reason. Fully offline. No API key. Deterministic. The same input always produces the same score, which is the only way to use something in a CI gate. What this experiment actually proves The gaps in the C-grade skills are invisible in normal review. If you read performance-optimization cold you would probably call it good, because the instructions are good. A human reviewer is not going to flag the absence of a boundary clause or notice that the Safety section is missing. They are going to read the content and nod. A linter does not read. It checks. And what it found here is that the most common quality gap in real-world agent skills is not bad instructions. It is the two or three structural signals the agent uses to decide when and whether to invoke the skill at all. That is a solvable problem. Now you have a number for it. Try it: pub.dev/packages/skillscore github.com/sayed3li97/skillscore addyosmani/agent-skills — the library used in this post Official authoring guides These are the primary sources skillscore's rules are drawn from. Each finding in the tool output cites one of them. Worth reading once if you author skills regularly. Anthropic — Agent Skills: Best Practices The canonical guide for Claude Code skills. Covers description quality (action verbs, when-clauses), conciseness (500-line body limit), progressive disclosure patterns, script documentation, and the overall structure of an effective SKILL.md. platform.claude.com/docs/en/agents-and-tools/agent-skills/best-practices Google Antigravity — Authoring Antigravity Skills (Google Codelabs) The official hands-on guide for Google Antigravity skills. Covers the Safety section requirement for skills that run commands, the boundary clause ("do not use when..."), and four levels of skill complexity from basic routing to procedural script execution. codelabs.developers.google.com/getting-started-with-antigravity-skills Agent Skills Open Standard — Specification The format specification that all agents implement: Claude Code, Codex, Antigravity, Gemini CLI, Cursor. Defines frontmatter fields, directory structure, optional folders, and progressive disclosure principles. agentskills.io/specification

8 hours ago

skillscore: a CLI that scores your AI agent's SKILL.md 0–100

A vague AI agent skill is worse than no skill at all — because the agent pays for it in context budget on every single turn, whether it uses it or not. Yet most of us write SKILL.md files by feel and ship them with zero feedback. So I built skillscore: a command-line tool that statically analyzes any SKILL.md and gives it a 0–100 quality score, a letter grade, and a list of fix-it findings — each one citing the official authoring guide it comes from. skillscore is an open-source Dart CLI that lints and scores AI agent skills (SKILL.md files) against the Claude, Codex, and Antigravity authoring guides. It runs fully offline, is deterministic, and exits with CI-friendly status codes. TL;DR 🎯 Scores any SKILL.md 0–100 with a letter grade and actionable findings across 7 categories. 📚 Rules are drawn from the official Anthropic (Claude), OpenAI (Codex), Google (Antigravity), and Flutter skill-authoring guides — and every finding cites its source. 🔌 Offline, deterministic, zero network calls. Same input → same score, every time. 🚦 Built for CI: --min-score 80, JSON output, and SARIF 2.1.0 that annotates pull requests. ⚡ Install: dart pub global activate skillscore → pub.dev/packages/skillscore Why I built this AI agent skills are quietly becoming a standard. A skill is just a folder with a SKILL.md — YAML frontmatter (a name and a description) plus a Markdown body of instructions — and optional references/, examples/, scripts/, and assets/ subfolders. Claude Code, Codex, Antigravity, Gemini CLI, and Cursor all read the same format. Here's the catch that most people miss: an agent keeps every skill's name and description in its context window permanently, so it can decide when to reach for one. A skill with a fuzzy description doesn't just fail to get used — it taxes every prompt and occasionally fires on the wrong request. The vendors all published authoring guides telling you how to avoid this: front-load triggers, write in the third person, state when not to use the skill, keep the body short, document your scripts. Good advice — scattered across four different documents, none of them enforceable. There was no eslint for skills. So I wrote one. What is skillscore? skillscore is a skill linter and SKILL.md validator that turns those authoring guides into 24 concrete, checkable rules. Point it at a file, a skill folder, or a whole monorepo, and it produces a score per skill: # Install (it's on pub.dev) dart pub global activate skillscore # Score a single skill — any name, any location skillscore path/to/SKILL.md # Score every skill in a tree skillscore path/to/skills/ The rules live in 7 weighted categories: Category What it checks A — Frontmatter validity --- delimiters, name format, description present B — Description quality states what + when, third person, front-loaded triggers, boundary clause C — Conciseness body length, no explainer bloat, no endless "or" chains D — Structure progressive disclosure, links one level deep, TOCs on long references E — Instruction quality anti-patterns, workflow checklist, feedback loop, code examples F — Content hygiene no rotting date references, forward-slash paths, consistent terms G — Safety & scripts a penalty (up to −15) when bundled scripts lack docs or a Safety section 100 points are distributed across A–F; category G only bites if your skill ships scripts or terminal commands. Profiles that exclude a vendor-specific rule are normalized back to 0–100, so a score means the same thing on every target. Does it actually work? Let's score a real one Here's skillscore run against a genuine skill from the Flutter team's public repo — flutter-add-widget-test/SKILL.md: flutter-add-widget-test (SKILL.md) Score: 90/100 Grade: A A Frontmatter validity 15/15 ██████████ B Description quality 21/25 ████████░░ C Conciseness & token economy 15/15 ██████████ D Structure & progressive disclosure 15/15 ██████████ E Instruction quality 14/20 ███████░░░ F Content hygiene 10/10 ██████████ G Safety & scripts no penalty WARNING E1_anti_patterns line 8 Body contains no explicit anti-patterns (no "do not", "never", or "avoid"). fix: Add explicit prohibitions, e.g. "Never share a WidgetTester across tests." INFO B5_boundary_clause line 3 Description has no boundary clause saying when NOT to use the skill. fix: Append a boundary, e.g. "Do not use for multi-screen integration tests." A genuinely good skill, and skillscore says so — but it also pinpoints the two things keeping it off a perfect score: it never tells the model what not to do, and its description doesn't state where the skill stops. Both are real, both are fixable in one line, and both come straight from the published guides. Want the rationale behind any finding? Ask: skillscore explain E1_anti_patterns It prints why the rule exists, the exact fix, and the source guide it's from. Built for CI A score you have to eyeball isn't a gate. skillscore is designed to live in a pipeline: # .github/workflows/skills.yml - name: Lint agent skills run: | dart pub global activate skillscore skillscore skills/ --min-score 80 --no-color --min-score 80 → the job exits non-zero if any skill dips below the bar. --format json → structured output for dashboards. --format sarif → valid SARIF 2.1.0 that uploads to GitHub code scanning, so findings annotate the exact lines in a pull request. Exit codes are pipeline-grade: 0 everything passed, 1 a gate failed, 2 a usage error. No flaky LLM in the loop, no network — the same skill always scores the same. How is this different from just asking an LLM to review my skill? skillscore Vendor schema check Markdown linter "Ask an LLM" Validates frontmatter ✅ ✅ ❌ ⚠️ Scores quality (discoverability, structure, instructions) ✅ ❌ ❌ ✅ Cites a source guide per finding ✅ ❌ ❌ ❌ Deterministic / reproducible ✅ ✅ ✅ ❌ Safe for a CI gate ✅ ✅ ✅ ❌ Offline ✅ ✅ ✅ ❌ An LLM review is great for nuance but non-deterministic — you can't gate a build on it. A schema check tells you the file is valid, not whether it's any good. skillscore fills the gap in the middle, and it pairs nicely with the other two. It's a library too The CLI is a thin wrapper over a public Dart API, so you can embed scoring in your own tooling: import 'package:skillscore/skillscore.dart'; void main() { final doc = SkillParser().parseFile('my-skill/SKILL.md'); final result = Scorer(RuleRegistry()).score(doc, Target.universal); print('${result.score}/100 ${result.grade}'); } FAQ What is an AI agent skill? A folder with a SKILL.md manifest (YAML frontmatter + Markdown instructions) that teaches an AI agent a repeatable task. Optional subfolders hold references, examples, scripts, and assets. The format is shared across Claude Code, Codex, Antigravity, Gemini CLI, and Cursor. Which agents does skillscore support? All of them — the SKILL.md format is shared. Score against one vendor with --target claude|codex|antigravity, or use the default universal profile, which a portable skill should pass everywhere. Is it really offline? Completely. No network calls at runtime, local files only, fully deterministic — the same input always produces the same score and the same finding order. Does my skill have to be named a particular way? No. skillscore is name-agnostic: the frontmatter name, the folder name, and the file name are independent, and even non-ASCII folder names work. Rule A2 will still tell you if the name field breaks the official format. What happens with malformed frontmatter? No crash. The relevant frontmatter errors are reported, every other rule that can still run does, and you always get a score. What's next v0.1.0 is live and the rubric is stable, but it's early. The roadmap: more vendor targets as new guides land, an autofix mode for the mechanical findings (forward slashes, missing TOCs), and a GitHub Action wrapper so CI setup is one line. The rule engine is deliberately simple — a new rule is one class plus one registration — so contributions are welcome, and every rule must cite the published guide it enforces. Try it dart pub global activate skillscore skillscore your-skill/ 📦 pub.dev: pub.dev/packages/skillscore 💻 GitHub: github.com/sayed3li97/skillscore sayed3li97 / skillscore Lint and score any AI agent SKILL.md against the official Claude, Codex, and Antigravity authoring guides — offline Dart CLI. skillscore — lint and score AI agent skills (SKILL.md) skillscore statically analyzes any AI agent skill — a SKILL.md manifest and its folder — and produces a 0–100 quality score, a letter grade and a list of actionable findings, scored against the official skill authoring guides from Anthropic (Claude), Google (Antigravity), and OpenAI (Codex). Offline, deterministic, CI-friendly. What is skillscore? skillscore is a skill linter / SKILL.md validator / agent-skill quality checker / AI skill scorer. Agent skills are an open standard — a folder with a SKILL.md (YAML frontmatter + Markdown body) plus optional references/, examples/, scripts/, and assets/ — used by Claude Code Codex, Antigravity, Gemini CLI, and Cursor. Because an agent keeps every skill's name and description in its context budget permanently, a vague or malformed skill is worse than no skill. skillscore catches exactly those… View on GitHub If you maintain skills, run it against your SKILL.md and tell me what score you get — and what it got wrong. I want the rules to reflect how people actually author skills, so findings you disagree with are the most useful feedback I can get. And if it saves you a context-budget headache, a ⭐ helps it reach other people building agents.

22 hours ago