Where Small Mistakes Matter

Abstract

Imitation learning typically treats every demonstration state as equally important, optimizing a behavior-cloning objective uniformly across the dataset. Yet not all states are created equal: in a small number of critical states, even a slight deviation from the expert action can lead to a failure, while in many other states such errors are recoverable and inconsequential.

We formalize state criticality in imitation learning as the worst-case degradation in downstream value induced by a bounded perturbation of the expert action at a given state. We then propose SCRIM, a practical State CRIticality Measure that estimates this quantity by training an adversarial disturbance policy and a value critic around a fixed imitation policy. SCRIM assigns each demonstration state a score reflecting how vulnerable the task outcome is to small, policy-relevant action errors.

We show that these scores can be used to reweight behavior-cloning losses, improving imitation at consequential states and increasing overall task performance. We further show that SCRIM enables data curation by selecting compact, high-value subsets of demonstrations that preserve performance while substantially reducing the amount of training data. We evaluate SCRIM across various simulated and real-world robotic manipulation tasks, and demonstrate that SCRIM improves policy training, enables efficient data selection, and provides interpretable insights into which moments in a demonstration are most critical for task success.

SCRIM finds the moments that matter

We compare SCRIM to four other ways of scoring states: distance from the gripper to the object, action magnitude, and two value-based measures (the variance of the policy's Q-function over actions, and the gap between the best and average Q-value). All scores are normalized to the same scale and plotted along a single demonstration.

Criticality doubles as a measure of task difficulty

SCRIM is computed with a single critic trained across all demonstrations of a task, so scores are comparable across trajectories and task variations.

On TurnFaucet, which spans ten faucet models, ranking the models by their mean SCRIM gives an intuitive easy-to-hard ordering: broad, forgiving handles score low, while handles that need a precise grasp and turn score high.

On a real robot

We learned SCRIM on a Franka Panda for two teleoperated tasks: StackCube (stack the red cube on the green cube, 70 demonstrations) and TapeHang (hang a roll of tape on a hook, 100 demonstrations). The base policies are fine-tuned from the \(\pi_{0.5}\) vision-language-action model, which uses a flow-matching action expert and is conditioned on three cameras, proprioception and a language instruction. The adversary is trained with SAC directly on the robot. It needed only 50 rollouts for StackCube and 150 for TapeHang.

SCRIM improves a real-world VLA

We use the hardware SCRIM scores the same way as in simulation: to reweight fine-tuning on the full dataset, and to curate a 25% subset of the demonstrations.

Success rate (%) over 25 real-robot rollouts per task and policy. Residual RL gets the same number of robot episodes as SCRIM's adversary (in parentheses).

SCRIM weighting lifts the fine-tuned VLA from 56% to 76% on StackCube and from 56% to 88% on TapeHang. Residual RL given the same robot episodes barely moves StackCube (60%) and collapses on TapeHang (4%). With a quarter of the demonstrations, SCRIM-selected subsets beat uniformly selected ones (68% vs 52% and 60% vs 48%), and even beat uniform fine-tuning on all the data. Because curation here drops whole demonstrations, it tests dataset compression rather than just resampling transitions.

Mean criticality flags fragile demonstrations

Because scores are comparable across trajectories, a demonstration's mean SCRIM grades its quality. A high mean marks a success that a small execution error would have turned into a failure; a low mean marks a robust one.

BibTeX

@misc{ciftci2026scrim,
  title  = {Where Small Mistakes Matter: Quantifying State Criticality for Imitation Learning},
  author = {Ciftci, Yusuf Umut and Kang, Joonwon and Sathya, Ajay Suresha and Bansal, Somil},
  year   = {2026}
}