StepBack reuses reasoning before a suspected error to cut repeated generation
StepBack proposes generating alternative reasoning paths from a suspected error onward while keeping the earlier steps. Researchers test the method against a strategy that produces complete answers repeatedly and chooses the most common result. Their comparisons cover seven reasoning and question-answering benchmarks, holding the candidate-answer count fixed. The approach changes inference-time sampling without retraining the model, while difficult early decisions remain a constraint.
Artificial Intelligence··Night
Earlier reasoning survives a new continuation
StepBack, an inference-time sampling method from North Carolina State University researchers, locates a suspected error in a draft answer and generates a replacement continuation. The preceding reasoning stays in place. The comparison method, self-consistency, repeatedly generates complete reasoning chains and chooses the most common final answer. StepBack also uses an answer vote, but begins with fewer drafts and reuses their prefixes when producing further candidates.[1]
Two signals choose the rollback point
A change in step-level token likelihood or a lower score from a process reward model selects the resampling point. With no explicit error signal, the method resamples the final answer step alone. In the main comparison, sixteen drafts supply their original answers and one continuation each, retaining thirty-two candidates. Steps are separated at blank lines. The generator’s weights do not require retraining.[1]
Tests cover reasoning and multi-hop questions
Seven benchmarks test mathematical reasoning and multi-hop question answering with models from the Qwen2.5 and Llama3.2 families. The authors report average accuracy near self-consistency with less generated text, with variation between model-task combinations. Hard problems can require revising an earlier decision left untouched by suffix resampling. Comparisons with random rollback and final-step-only resampling also examine the value of locating the error.[1]
