TIGER-AI-Lab/WebsiteBench

WebsiteBench from TIGER-AI-Lab: Agents are given a production-grade website accessible via a browser and are required to reverse-engineer a full-stack version that closely matches its appearance and behavior. The benchmark includes 70 websites and 46,916 hidden test cases, with each task containing 302 to 2,016 cases. A case passes only if all functional checks succeed and the normalized SSIM for every visual checkpoint reaches 0.99. The task score is the proportion of passed cases. The tasks are hosted on Harbor Hub and the Hugging Face dataset.

WebsiteBench from TIGER-AI-Lab: Agents are given a production-grade website accessible via a browser and are required to reverse-engineer a full-stack version that closely matches its appearance and behavior. The benchmark includes 70 websites and 46,916 hidden test cases, with each task containing 302 to 2,016 cases. A case passes only if all functional checks succeed and the normalized SSIM for every visual checkpoint reaches 0.99. The task score is the proportion of passed cases. The tasks are hosted on Harbor Hub and the Hugging Face dataset.

Why It Matters Much harder than having a model write a page from a screenshot: the agent must independently discover interactions and functionality, with both appearance and behavior needing to pass hidden tests.

Stars 293 (+42 / 24h)

Reasons

  • 42 new stars in 24 hours (indicating attention only)
  • The scoring rules are highly specific, and the task definitions include identifiers to prevent training data contamination

Usage

  • Use pip install websitebench to download the tasks; after installing Docker and Harbor, use harbor run to specify the dataset and agent to run all tasks

Risks

  • The reference websites are copies of third-party sites and are for non-commercial research use only
  • Running all tasks requires a Docker environment and a large number of model calls

Open

GitHub

Explore the latest AI Radar updates