A web agent environment suite initiated by the University of North Carolina at Chapel Hill and Microsoft: Using coding agents (such as Claude Code / Codex) to create 153 local Docker images based on real websites, replicating interfaces, content, and functional flows like login, shopping carts, and bookings. These are manually reviewed for visual fidelity, functional correctness, and task validity; the environment expands as new tasks require additional features. It includes 3,518 tasks (including multi-step and post-login flows), with each website having a validator that provides 0/1 rewards, and a control interface capable of resetting the database between episodes. The authors use it for reinforcement learning on 4B-scale GUI agents, then evaluate them on websites and tasks unseen during training.
Likes 19 · Code Stars 46
Reasons
- The authors state that WebHarbor-4B (Qwen3-VL 4B, starting from OpenWebRL-4B-SFT), trained only within the WebHarbor synthetic environment, achieves an average of 75.8 on WebVoyager and Online-Mind2Web; the model card lists scores of 81.2 for WebVoyager and 70.4 for Online-Mind2Web. None of the 15 WebVoyager websites are involved in training.
- Addresses long-standing issues with evaluating and training on real websites: changing content, CAPTCHAs, regional restrictions, and login walls; local images offer stability and reproducibility, with the authors stating that database resets occur in sub-second timeframes, making them suitable for large-scale RL episodes.
- Ready to use: A single Docker Hub image wildawm/webharbor contains all 153 sites (approx. 20 GB); tasks are provided in Harbor format on Hugging Face ChilleD/WebHarbor-Tasks (with splits such as 2,288 training tasks and 306 test tasks), and the model ChilleD/WebHarbor-4B can be launched on a single GPU; it received 19 likes and 46 code stars on Hugging Face that day (indicating attention only).
Usage
- Start the environment: export WEBSYN_CONTROL_TOKEN=<random string of at least 32 characters>, docker run -e WEBSYN_CONTROL_TOKEN -p 8101:8101 -p 40000-40152:40000-40152 wildawm/webharbor:latest; agents access ports 40000–40152 on localhost; use curl -H "Authorization: Bearer $WEBSYN_CONTROL_TOKEN" -X POST http://localhost:8101/reset/amazon to reset a single site, or /reset-all to reset all sites.
- Run tasks: uv tool install harbor==0.24.0, then in the repository's harbor/ directory run harbor run --repo https://huggingface.co/datasets/ChilleD/WebHarbor-Tasks -d [email protected] -a webharbor_agents.openwebrl:OpenWebRLWebHarbor -m openai/<model name>; the model under test connects via an OpenAI-compatible interface.
- Start the model: pip install sglang==0.5.6.post2, python -m sglang.launch_server --model-path ChilleD/WebHarbor-4B --context-length 32768 … (the single-GPU 32K launch command provided in the model card); the interface is available at localhost:30000/v1.
Risks
- The images mimic real brand websites such as Amazon, Chase Bank, and American Express; neither the repository README nor the model card specifies a license, so confirmation is required before commercial use or redistribution.
- Anonymous pulls from Docker Hub are subject to rate limits; docker login is required to pull many single-site images; the optional agent reviewer defaults to calling gpt-5.6-sol, which requires your own API key.
- The mimicked sites are generated by models and manually reviewed; differences from the real sites can only be assessed via the review details in the paper; scores are as reported by the authors.