SPPO: Sequence-Level PPO for Long-Horizon Reasoning

This work identifies instability in token-level PPO for long-horizon reasoning and proposes a sequence-level optimization view with lower variance and lower memory overhead.

Recommended citation: @misc{wang2026spposequencelevelppolonghorizon, title={SPPO: Sequence-Level PPO for Long-Horizon Reasoning Tasks}, author={Tianyi Wang and Yixia Li and Long Li and Yibiao Chen and Shaohan Huang and Yun Chen and Peng Li and Yang Liu and Guanhua Chen}, year={2026}, eprint={2604.08865}, archivePrefix={arXiv}, primaryClass={cs.AI}, url={https://arxiv.org/abs/2604.08865}, }
Download Paper