Estimating physics from images: what real measurements reveal (opens in a new tab)
A useful reality check for anyone turning images into simulation assets: shape is only the beginning. Read the measured comparisons, then linger over the LEGO brick whose friction changes with the surface beneath it.
In brief: Estimating physics from images: what real measurements reveal
ProjectSim evaluates physical properties inferred from images against measurements of object mass, friction, hardness, and rebound. Its estimates outperform simple density or constant-value baselines for mass and friction. The more revealing experiment asks how friction should be represented: fitting separate object and surface coefficients captures known contacts better than assigning one coefficient to each object. That fit uses the same measurements for fitting and scoring, so it is a diagnostic rather than evidence of generalization. The evaluations also cover different, sometimes small samples; they do not establish that a robot trained with these assets performs better.