← Back to Benchmarks
simmediumroboticsmetric · varies
Beyond Binary Success: Sample-Efficient and Statistically Rigorous Robot Policy Comparison
Description
Generalist robot manipulation policies are becoming increasingly capable, but are limited in evaluation to a small number of hardware rollouts. This strong resource constraint in real-world testing necessitates both more informative performance measures and reliable and efficient evaluation procedures to properly assess model capabilities and benchmark progress in the field. This work presents a novel framework for robot policy comparison that is sample-efficient, statistically rigorous, and app