WebsiteBench from TIGER-AI-Lab: Agents are given a production-grade website accessible via a browser and are required to reverse-engineer a full-stack version that closely matches its appearance and behavior. The benchmark includes 70 websites and 46,916 hidden test cases, with each task containing 302 to 2,016 cases. A case passes only if all functional checks succeed and the normalized SSIM for every visual checkpoint reaches 0.99. The task score is the proportion of passed cases. The tasks are hosted on Harbor Hub and the Hugging Face dataset.
Why It Matters Much harder than having a model write a page from a screenshot: the agent must independently discover interactions and functionality, with both appearance and behavior needing to pass hidden tests.
Stars 293 (+42 / 24h)
Reasons
- 42 new stars in 24 hours (indicating attention only)
- The scoring rules are highly specific, and the task definitions include identifiers to prevent training data contamination
Usage
- Use
pip install websitebenchto download the tasks; after installing Docker and Harbor, useharbor runto specify the dataset and agent to run all tasks
Risks
- The reference websites are copies of third-party sites and are for non-commercial research use only
- Running all tasks requires a Docker environment and a large number of model calls