A video tool plugin for DeepSeek Harness: compresses a video into a single storyboard image with 9–16 cells based on shot transitions, including cell numbers and timestamps, a volume bar at the bottom, and subtitles or speech transcriptions aligned by cell. After viewing, the model can request zoomed-in frames for specific cells.
Star count: 40
Reasons
- Solves the high cost of letting models "watch" videos: the author claims one image costs about 384 tokens, allowing a 5-minute video to be viewed in full with a single image, whereas a one-frame-per-image approach covering the same duration would require 3,072–6,144 tokens.
- Cut points are detected via HSV color changes, selecting the sharpest frame from three candidates per cell; fast-cut segments are allocated more cells. The author claims all 9 hard cuts were detected in their custom test video, while ffmpeg's scdet with default thresholds found only 4.
- For long videos, use from / to to view 2–5 minutes at a time, avoiding spoilers for later plot points; textOnly pagination reads transcriptions without generating images.
- No runtime dependencies other than ffmpeg, and no model weights included; the repository contains 154 tests.
Usage
- Requires Node 20+, with ffmpeg / ffprobe in the PATH. For standard profiles, use dsh plugin --profile web add dsh-video-storyboard; for desktop profiles, use node scripts/install-dev.mjs from the repository, then restart DSH after installation.
- Call video_storyboard({ input: "local video path" }) in the conversation; use video_zoom when detailed inspection is needed. From the command line, you can first run node bin/storyboard.mjs your-video.mp4 --cells 9 --text to see what the model will receive.
- If the video lacks subtitles, configure a separate OpenAI-compatible /audio/transcriptions speech-to-text service (set DSH_ASR_KEY); otherwise, only visuals and the volume bar will be generated.
Risks
- The model used must support image input; even with more cells, there is no guarantee that every shot will be seen, and the author notes that rapid pans may be mistakenly detected as cut points.
- Passing web links such as Bilibili requires installing yt-dlp and providing cookies.txt, which involves your own login credentials and platform terms of service; speed metrics are based on the author's self-tests.