SkillEvaluator is divided into three layers: the first layer checks whether skills are safe, properly formatted, and compatible with various agents, invoking SkillSpector and Gitleaks for scanning; the second layer uses vectors to detect duplicate content within and between skills; the third layer has agents complete tasks in a sandbox both with and without the skill, comparing the results. The first layer can operate entirely without models or API keys, achieving a score of 86.2 using built-in examples; it can also use NVIDIA Build's free quota to run model-based scoring. It is part of NVIDIA's certified skill workflow and includes an arXiv paper.
Why it matters Skills are increasingly installed, yet few people verify whether they actually help; it combines static checks, duplicate detection, and controlled experiments in a sandbox into a single command, and can be integrated into CI.
Stars 1703 (+122 / 24h)
Reasons
- 122 new stars in 24 hours, total stars 1703 (indicating attention only); Apache-2.0 license
- The README clearly specifies the requirements for each layer (scanners, models, Docker sandbox), marking layers that cannot run as incomplete
Usage
- After
pip install skillevaluator, runskillevaluator validate ./my-skill --tiers 1for checks without keys; runskillevaluator ci initin a GitHub repository to automatically open a PR integrating CI
Risks
- The third layer requires model services, supported agents, and a Docker sandbox; model scoring sends skill documentation to the selected model service