Composo open-sources DishwasherBench, a dishwasher-loading benchmark
What happened
On October 13, Composo open-sourced DishwasherBench 0.1.0, which tests large language models on loading dishes using a physical simulation of a real dishwasher. Humans can play it in a browser, and models can operate it through MCP. Two models ran six official simulations across three levels; both beat a cautious scripted player on complete levels, but the records are not enough for ranking and there is no human baseline. The simulation was calibrated against one real wash, and scoring can judge loading and the orientation of cups and pots, but not yet whether stacking plates and utensils affects cleaning.
Written by AI from the coverage · updated 1 hour ago
Coverage
Follow the reports to see the story from different sides.
- Hacker News front pageDishwasherBench – Can Frontier Models Stack a Dishwasher?
Composo 开源 DishwasherBench 0.1.0，用真实洗碗机的物理仿真测试大语言模型装载餐具的能力，支持人类在浏览器中游玩及模型通过 MCP 操作。两个模型部署在三个关卡完成了六次正式仿真，两者在完整关卡均超过谨慎脚本玩家，但这些记录不足以排名，且没有人类基线。仿真仅用一次真实清洗校准，目前评分能判断装载和杯、锅等容器的朝向，尚不能可靠判断盘子与餐具的堆叠是否影响洗净。
Heat over time
Not enough continuous observations to draw a trend yet.