A smaller response window can make a masked diffusion language model commit to a wrong answer even when a correct solution fits.
Xiang Li and colleagues delayed answer-tail commitments at the same budget. They recovered correct answers in 43.2% of 155 selected cases where low-budget decoding failed but high-budget decoding succeeded. A matched order control recovered 30.3%.
Their HiLo-RD trains on the same model's larger-window hidden states to improve smaller-window decoding. The builder's problem is when the model commits, as well as what it predicts.
https://arxiv.org/abs/2610.36901v1