Learning the shape of a push

PushT looks almost trivial: use a circular pusher to move a T-shaped block into a matching target. The difficulty is hidden in contact. A push that translates the block from one angle can rotate it away from the goal from another. A plan that is geometrically short can be dynamically impossible.

The controller therefore cannot succeed by drawing a path and hoping the object follows it. It has to learn what a push does: when contact begins, how motion transfers through an off-center collision, when rotation dominates translation, and when the current plan should be abandoned.

The strongest run rose from 68% at the first frozen checkpoint to 100% after 90 minutes, then closed at 99%. This is not a single lucky trajectory: every point is the success rate over the same 100 unseen initial states.

We also changed what the system could observe. Supplying motion directly produced the fastest rise; inferring it from state history improved more gradually; starting from images was less stable. The plot reports every scheduled snapshot, including regressions and plateaus.

The curve says the improvement generalizes. A single held-out start shows how the behavior changes. We replayed one held-out start with all six frozen versions of the same system.

It remains stuck at 30 and 60 minutes, with only 50.6% and 54.7% overlap. At 90 minutes it crosses the episode threshold for the first time; the finishing contact continues to change through 120 and 150 minutes, reaching 98.2%. The final two checkpoints match, so the plateau remains visible rather than edited out.

The world model and the planner do different jobs. The model answers a local question: given this state and this push, what state should follow? The planner composes those answers to search for a useful sequence. If the model predicts the wrong contact, a more expensive search only finds a more confidently wrong plan.

The controller continually re-anchors itself to the real observation. It plans through the executable model, acts for a short horizon, compares the predicted transition with the observed one, and replans when the difference becomes meaningful. An approximate model can therefore be operationally strong: it only has to be trusted long enough to choose the next useful action.

The six clips expose that division. Later checkpoints do not simply “try harder.” They support different contact choices, different corrections, and eventually a more complete final alignment from the same initial state.

The bar chart places that result alongside four published PushT scores. Against the public LeRobot Diffusion Policy checkpoint specifically, the difference is 33.6 percentage points.

PushT has unusually clear boundaries: one object, one target shape, a clean 2D plane, a fixed action space, and a short horizon. That makes it a useful test of contact learning, but not yet a test of whether the same system can keep learning across changing objects, goals, sensors, and dynamics.

Expanding the task means adding unfamiliar geometries, noisy or partial observations, moving goals, and sequences of tasks where old knowledge should help without becoming a brittle prior. The system should reuse what transfers and revise what does not, rather than relearning every world from zero.

Speed matters just as much. The three-hour trace is promising, but the loop should become more selective about which evidence to collect, reuse learned contact primitives, evaluate revisions in parallel, and reach a trustworthy model with far fewer real interactions. The goal is not a larger PushT score. It is a shorter path from surprise to a model that can act.