Reset Dependence Is Embodied: Training Protocol Changes Which Morphologies Appear Optimal

Kevin Qiu 1 Zhao Yang 2 Marek Cygan 1
1 University of Warsaw 2 Vrije Universiteit Amsterdam
NeurIPS 2026 · Evaluations & Datasets Track

Changing only the reset rule used to train controllers changes which of 100 evolved robot bodies look best, and how much a body loses without fall resets differs across evolutionary families.

Reset dependence is embodied. The Floor A body in illustrative retrains under each reset rule, in real time. It walks under Standard (τ = 0.20), flails without moving forward under Reset-free (τ = 0), and under Strict (τ = 0.50) its episode ends at step 9 (paper, Figure 1).

In short

What we tested

We train PPO controllers from scratch for 100 bodies of the UNIMAL design space, evolved by DERL on flat ground (Floor), on variable terrain (VT) and for pushing a box (MVT), all on one task: walking forward on flat ground. During training, an episode ends when the torso falls below a fraction τ of its starting height: Standard (τ = 0.20), Strict (τ = 0.50) or Reset-free (no fall cutoff; the 1,000-step time limit stays). With three seeds per rule that makes 900 controllers, and one common evaluator scores them all. The results also hold without the bodies whose cutoffs do not act as intended (paper, Section 3).

Three panels sketch the states visited during training under Strict (τ = 0.50), Standard (τ = 0.20) and Reset-free (τ = 0) training, each as an oval of torso height against pose with the start state at the upper left. Under Strict training, moderate drops cross the cutoff and trigger a reset, so the body rarely leaves the neighborhood of its start state. Under Standard training, the body walks forward, occasionally falls below the lower cutoff and resets. Under Reset-free training there is no height rule, the trajectory winds through walking and post-fall states, and the 1,000-step limit still ends every episode.

Strict training keeps rollouts near standing states, while Reset-free training exposes bodies to post-fall states the controller must handle. Each panel sketches the states one rule trains on. Reaching the region below the dashed cutoff ends an episode, except under Reset-free (paper, Figure 2).

Changing the rule reorders the bodies

Two scatter plots of the 100 bodies' ranks. Left: ranks after Reset-free training with seed 42 against seed 44; the points lie close to the diagonal (Spearman rho = 0.76). Right: ranks after Reset-free and after Strict training, both with seed 42; the points scatter across the whole square (rho = 0.22).

Changing the reset rule reorders the bodies far more than changing the seed. Each dot is one body, ranked by per-step reward (1 = best). Two seeds of the same rule rank the bodies alike (left), but the same seed under Reset-free and Strict training does not (right) (paper, Section 4.2 and Table 2).

Families respond differently

Three panels by family (Floor, VT, MVT) and training protocol (Standard, Strict, Reset-free): per-step reward, distance per evaluation episode, and per-step reward relative to Standard training. After Reset-free training, Floor bodies keep on average 0.55 of their Standard reward and VT bodies 0.84. A few bodies with near-zero Strict values come from cut-off runs.

Under Reset-free training, the whole Floor distribution shifts down relative to VT, not just a few bodies. Per-step reward (left), distance (middle) and reward relative to Standard training (right); points are bodies, and large markers show family means with 95% bootstrap CIs (paper, Figure 3).

The difference holds in permutation tests over bodies and without an evaluation cutoff, and on 24 bodies, Floor bodies gain the most from Strict over Reset-free training under both PPO and SAC (paper, Sections 4.4 to 4.6).

Some bodies walk under every rule

Floor B, from the same family as Floor A, walks under all three rules, as in all nine of its recorded runs, so sensitivity to the reset rule varies from body to body (paper, Figure 13).

An audit for co-design benchmarks

  1. State the reset rule and check that it acts on every body. Compare each body’s cutoff with its resting torso height and with the drop from its reset pose.
  2. Retrain the top bodies under one looser and one stricter threshold, and score every controller with a common evaluator (120 runs for 20 bodies and three seeds).
  3. Compare rank agreement with seed noise, and report bodies that stay strong under every rule (paper, Section 5).

Code and run records

One repository holds the training and analysis code, the run records behind the paper’s results, and a CPU script that recomputes 2,401 values behind the paper in a few minutes, under the MIT and CC BY 4.0 licenses with Croissant metadata: github.com/kevinxqiu/reset-dependence, with the run records attached to release v1.1.2.

Abstract

Robot co-design benchmarks rank bodies after training their controllers under a fixed episode protocol, but that protocol can change the ranking itself. Consequently, two studies can select different bodies from the same corpus even when they use the same task and evaluator. We study one part of that protocol, the reset rule, which ends a training episode when the robot’s torso falls below a height threshold. We train PPO controllers for 100 DERL-evolved UNIMAL bodies with this threshold at 20% of each body’s starting torso height (Standard), at 50% (Strict), or with no threshold (Reset-free), and score every controller with the same evaluator. The threshold changes which bodies look best: the rankings after Strict and Reset-free training correlate at only 0.21, against about 0.89 if seed noise were the only difference. Removing fall resets costs some bodies far more than others. Bodies evolved on flat ground lose 45% of their per-step reward relative to Standard training, and those evolved on variable terrain only 16%. Permutation tests over bodies support this difference, and on 24 bodies, flat-ground bodies gain the most from Strict over Reset-free training under both PPO and SAC. Choosing bodies by their worst reward across the three protocols costs little: in sample, they keep at least 91% of the reward of each protocol’s best bodies, and over 80% averaged across held-out seeds. Co-design benchmarks should state their reset rule, test whether their rankings depend on it, and report bodies that stay strong across protocols.

BibTeX

BibTeX
@inproceedings{qiu2026reset,
title     = {Reset Dependence Is Embodied: Training Protocol Changes Which Morphologies Appear Optimal},
author    = {Qiu, Kevin and Yang, Zhao and Cygan, Marek},
booktitle = {Advances in Neural Information Processing Systems (Evaluations and Datasets Track)},
year      = {2026}
}