Publications

SPPO: Sequence-Level PPO for Long-Horizon Reasoning

ACL 2026 Oral · Co-first Author

Tianyi Wang*, Yixia Li*, Long Li, Yibiao Chen, Shaohan Huang, Yun Chen, Peng Li, Yang Liu, Guanhua Chen

Reformulates long-horizon reasoning as a sequence-level contextual bandit with a decoupled scalar value function to improve stability and reduce memory cost.

arXiv GitHub

DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay

Under Review

Long Li, Zhijian Zhou, Tianyi Wang, Weidi Xu, Zuming Huang, Wei Chu, Zhe Wang, Shirui Pan, Chao Qu, Yuan Qi

Introduces Dynamic Jensen-Shannon Replay to preserve diversity in reinforcement learning through verifiable rewards mechanism.

arXiv

Anchored Policy Optimization: Mitigating Exploration Collapse via Support-Constrained Rectification

ICML 2026 · Co-first Author

Tianyi Wang*, Long Li*, Hongcan Guo, Yibiao Chen, Yixia Li, Yong Wang, Yun Chen, Guanhua Chen

Introduces support-constrained rectification for RLVR to mitigate exploration collapse and improve both Pass@1 and response diversity.

arXiv GitHub