Microsoft Mage team's 4B computer operation model: After viewing desktop screenshots, it decides at each step whether to operate the interface using mouse and keyboard (computer_use, 11 types of actions) or to run shell / Python in the terminal (bash). A single model completes cross-application desktop tasks without an external scheduler. It is trained from an intermediate checkpoint of Mage-VL 4B (Mage-ViT plus Qwen3-4B).
Upvotes 90 · Likes 91 · Downloads 233
Reasons
- Officially reported OSWorld-Verified 66.0% (361 questions, excluding Google Drive types, 50 step budget, averaged over two full evaluations), with 87.5% for operating system tasks and 62.7% for multi-application workflows; in the same table, ScaleCUA-Qwen3.5-9B 68.7, Qwen3.6-27B 63.9, GPT-5.4-mini 69.6, and human reference 72.4.
- It divides labor: Structured modifications for spreadsheets, documents, and presentations are made directly in the terminal using openpyxl / python-pptx, while mouse clicks are reserved for canvas editing and viewing interface states. The official demo showed a LibreOffice Calc task (merging cells, adding a bar chart, saving as .xlsx) completed in 39 seconds and 6 steps, whereas Qwen3.6-27B, which only clicks the interface, took 9 minutes and 56 seconds and 39 steps.
- The training recipe is fully documented: Five stages in sequence are GUI mid-training, GUI agent SFT, programming / terminal SFT, mixed SFT, and online GRPO reinforcement learning, synthesizing a total of 293,806 tasks. Officially, reinforcement learning outperforms mixed SFT by 10.16 percentage points while generating 25.4% fewer output tokens. The repository was created on 10 30, and it was not included in previous issues.
Usage
- Service: Use the official Docker image xiaoyizhang/mage-vl-cowork:0.1.0 (which includes SGLang 0.5.2); mount the weights directory to start the service. Alternatively, run python -m sglang.launch_server --model-path microsoft/Mage-VL-Cowork --trust-remote-code --context-length 32768 --tool-call-parser qwen25 yourself, and set SGLANG_MAX_PIXELS=4000000 to preserve the details of 1920×1080 screenshots.
- The system prompt must be used exactly as defined in the model card's MAGE_HYBRID_SYSTEM_PROMPT, which defines the computer_use and bash tools, with coordinates normalized to 0–999. At each step, the model outputs <think> reasoning, a single Action, and <tool_call>; sampling uses temperature 0.7, top_p 0.8, and max_new_tokens 4096.
- To run full tasks, integrate the MAGEHybridAgent from the repository's osworld_eval/ directory: Keep the most recent 3 screenshots, and collapse earlier steps into a one-sentence summary. Offline inference with Transformers requires transformers>=4.57.3 and trust_remote_code=True.
Risks
- It executes arbitrary shell commands and interface operations. Officially, it is required to run only in isolated, disposable containers or virtual machines, without mounting personal keys, browser cookies, or cloud credentials, and manual confirmation is required before irreversible operations such as payments or logins.
- All scores are self-reported by the official team, 66.0 using their own harness and averaging two runs. The 58.6 score for the web task WebVoyager-Static is from the SFT version, which is lower than the browser-specific Fara1.5-4B (68.0). The model card only contains scores for Linux desktops (OSWorld).
- The architecture is a custom llava_onevision2, requiring trust_remote_code to be enabled; it cannot be loaded as a standard Qwen3-VL. The inference window is trained for 32K. The bf16 weights are 9.48 GB, and VRAM requirements are not specified in the model card; the technical report is currently a PDF on GitHub.