molly554/whiteboard-ai-tutor

This project enables large language models to output JSON drawing instructions based on a fixed prompt, which the browser plays back as handwriting, line drawing, and erasing. Students can interrupt at any point, and the application sends the current state of the whiteboard back to the model. The author tested six models across Claude, Gemini, and DeepSeek using the same prompt set, covering eight middle school science and economics scenarios. The extracted whiteboards were blind-evaluated by models not included in the tested list. The author claims that Gemini 4 Argon 11 passed all blocks; DeepSeek V4 Pro passed 5 out of 10 blocks, and V4.1 Flash passed 4 blocks. When thinking mode was disabled for the latter, 7 out of 24 calls returned instructions with missing fields, resulting in blank whiteboards. However, the cost per lesson for both was only a fraction of a cent to less than one cent.

This project enables large language models to output JSON drawing instructions based on a fixed prompt, which the browser plays back as handwriting, line drawing, and erasing. Students can interrupt at any point, and the application sends the current state of the whiteboard back to the model. The author tested six models across Claude, Gemini, and DeepSeek using the same prompt set, covering eight middle school science and economics scenarios. The extracted whiteboards were blind-evaluated by models not included in the tested list. The author claims that Gemini 4 Argon 11 passed all blocks; DeepSeek V4 Pro passed 5 out of 10 blocks, and V4.1 Flash passed 4 blocks. When thinking mode was disabled for the latter, 7 out of 24 calls returned instructions with missing fields, resulting in blank whiteboards. However, the cost per lesson for both was only a fraction of a cent to less than one cent.

Why It Matters Provides a reproducible comparison for those looking to build AI-driven lectures: DeepSeek is cheap and generates the first output quickly, but often fails to follow the drawing protocol when thinking mode is disabled.

Star Count 61

Reasons

  • Timing, usage, costs, and raw outputs for all 288 calls are available in the repository, and reviewed whiteboards can be replayed stroke-by-stroke online
  • The author highlights limitations upfront: each model was evaluated on only about 11 blocks, making it difficult to distinguish mid-tier rankings; the prompt was written in only one Chinese variant

Usage

  • To view results without an API key: clone the repository, run python serve.py, and open the local gallery. To run it yourself, set OPENROUTER_API_KEY and DEEPSEEK_API_KEY. The author states that a full run costs approximately 4 USD

Risks

  • Scoring is performed via blind evaluation by models, with an additional 18 blocks reserved for manual review; conclusions apply only to this specific prompt and these eight scenarios
  • This is not a hosted educational application; the player replays recorded lessons

Open

GitHub

Explore the latest AI Radar updates