A robotics AI benchmark for autonomous skilled work, and the first benchmark for fully autonomous skilled work. A model gets a robotic arm, a bench of powered tools, and a broken phone. No script, no human hands. It fixes the phone to standard, or it does not.
Subjects are real models doing their best. There are no scripted policies and no partial credit for effort: the phone is judged exactly as a customer would judge it.
The agent drives a dexterous arm on a bench of powered tools: a screwdriver with loadable bits and torque control, opening tools with agent-set force and temperature, cleaning and diagnostic tools, and a display-touch system that taps and swipes the screen it sees through a mirror feed. Every device enters in a recorded entry condition.
Runs are scored on time, efficiency, and accuracy, gated by quality tiers. Normal: the repair works and nothing is damaged. High: the device and its data are respected, standard tests only, nothing restored or updated unless asked. Max: no sign anyone was ever inside, down to fingerprints. A safety rule always outranks a tier.
Every run must persist a full transcript of every action, authored by the benchmarked model. No transcript, no score. Scorecards are published with the failures next to the passes.
Scenarios are real repairs parameterized from published repair guides, down to every screw.
Battery, display, and camera work on an iPhone 14 Pro Max, and a diagnosis-first scenario on an iPad Pro 11-inch M1 that does not boot: find the fault with whitelisted diagnostics only, declare it, then repair it.
Unedited outcomes, in the order they happened. Repairs that completed but missed the standard are listed as what they are.
Repair completed end to end. Failed at every tier on a single latched damage check: four screws stripped in the first minutes by a bit-name guessing trap in the tool interface, later fixed in the simulator with regression tests.
Repair completed with zero damage events. Failed on paperwork and cosmetics: one handling smudge, and a diagnostic call rejected because the agent used the part name the simulator itself had shown it.
The suite only clean sweep. Correct diagnosis declared before any repair: a failed, mildly swollen battery cell. Replaced it on pull tabs only, no heat near the pack. Normal, High, and Max all passed.
All runs operated by Muse Wonder (via Muse harness).