A local tool that lets coding agents "hear" sound: it turns audio into evidence agents can reason about (text descriptions, match scores, timestamps, full transcriptions), and can synthesize sound effects via code, iterating based on listening.
Stars 3050 (+108 / 24h)
Reasons
- Commands like describe / match / find / timeline provide insights on what the sound resembles, how well it matches a description, the most suitable segments in long recordings, and a second-by-second "transcription" of the entire video along with audio-visual synchronization status
- critique scores sound effects for code-generated promotional videos and provides timestamped modification suggestions, referencing 7 professionally released videos curated by the author
- forge creates sound effects in code using physical synthesis recipes, evolving the search after scoring by an internal reviewer; the author states that the 0.4 version has a sound effect library of 252 items, adds launch presets, and places each sound effect according to the soundtrack and edit points
- Provides CLI, Python API, and MCP server; the author documents how each metric was tested in docs/VALIDATION.md and separately explains which judgments are unreliable in LIMITS.md
Usage
- Requires Python 3.10+ and ffmpeg; after git clone, create a venv, install the CPU version of torch first, then run pip install -e ".[mcp,llm,dev]", and use python -m earshot doctor --load for a self-check
- Integration with Claude Code: claude mcp add earshot --scope user -- <venv>/bin/python -m earshot mcp (the MCP approach keeps the model resident, avoiding reloading for each command)
- Two skills (earshot, real-sound) are included under the repository's skills/ directory; copying them to ~/.claude/skills/ allows agents to use them according to the workflow
Risks
- First use downloads two open models (the author states approximately 1.1 GB); each CLI call requires reloading the model, which the author states takes 20–40 seconds
- The author acknowledges that synthesizing sound effects optimizes for the model's own scoring, so model checks may be "gamed"; the realism model is easily fooled and cannot hear frequencies above 14 kHz or speech content
- Reference recordings are copyrighted and not included in the repository; forge's --refset requires users to provide their own reference materials; the code is MIT licensed, while the two models have their own respective licenses