Papperoni

2026

HEAL: Hindsight Entropy-Assisted Learning for Reasoning Distillation

Wenjing Zhang, Ping Chen, Jiangze Yan, Jieyun Huang, Yi Shen, Shuming Shi, Ning Wang, Zhaoxiang Liu, Kai Wang, Shiguo Lian

Open PDF Google Scholar

citations

Cite Score

0

AI summary

This paper introduces Hindsight Entropy-Assisted Learning (HEAL), an RL-free framework with Guided Entropy-Assisted Repair (GEAR), Perplexity-Uncertainty Ratio Estimator (PURE), and Progressive Answer-guided Curriculum Evolution (PACE) to distill reasoning capabilities from large models to smaller ones, outperforming SFT distillation and other baselines on multiple benchmarks.

Main Contributions

Proposed HEAL, an interventionist framework applying ZPD principles to LRM distillation, actively repairing failed reasoning trajectories via GEAR.
Designed PURE, a ratio-based filtering metric for pedagogical refinement, decoupling genuine reasoning breakthroughs from spurious shortcuts to ensure high-quality lesson plans for distillation.
Introduced PACE, a three-stage curriculum distillation strategy, and validated its effectiveness across multiple datasets and model scales.
Achieved an average accuracy of 61.68% on the Qwen2.5-14B-Instruct backbone, surpassing standard SFT by a substantial 10.69% margin.
Demonstrated the framework's ability to unlock the teacher's latent reasoning potential and bridge the gap toward the teacher's true capabilities.

Abstract

Distilling reasoning capabilities from Large Reasoning Models (LRMs) into smaller models is typically constrained by the limitation of rejection sampling. Standard methods treat the teacher as a static filter, discarding complex "corner-case" problems where the teacher fails to explore valid solutions independently, thereby creating an artificial "Teacher Ceiling" for the student. In this work, we propose Hindsight Entropy-Assisted Learning (HEAL), an RL-free framework designed to bridge this reasoning gap. Drawing on the educational theory of the Zone of Proximal Development (ZPD), HEAL synergizes three core modules: (1) Guided Entropy-Assisted Repair (GEAR), an active intervention mechanism that detects critical reasoning breakpoints via entropy dynamics and injects targeted hindsight hints to repair broken trajectories; (2) Perplexity-Uncertainty Ratio Estimator (PURE), a rigorous filtering protocol that decouples genuine cognitive breakthroughs from spurious shortcuts; and (3) Progressive Answer-guided Curriculum Evolution (PACE), a three-stage distillation strategy that organizes training from foundational alignment to frontier breakthrough. Extensive experiments on multiple benchmarks demonstrate that HEAL significantly outperforms traditional SFT distillation and other baselines.

Citation Graph

Loading graph...

References [32]

Sort:

Filter:

[1]Deepseek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Deepseek Ai - 2025

3 papers in library cite

Finally the RLVR paper!

[2]Measuring Mathematical Problem Solving With the MATH Dataset

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, Jacob Steinhardt - 2021

12 papers in library cite

Good idea in creating the dataset but it's just a regular and uninteresting dataset description paper

[3]Let's Verify Step by Step

Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, Karl Cobbe - 2023

5 papers in library cite

Fun read but nothing really surprising in terms of methodology. Just proving PRMs are better than ORM (again, not surprising)

[4]GPQA: A Graduate-Level Google-Proof Q&A Benchmark

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, Samuel R. Bowman - 2024

4 papers in library cite

Even though it's small, it's very well made!

[5]LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

Naman Jain, King Han, Albert Gu, Wentao Li, Fanjia Yan, Tong Zhang, Shijie Wang, Armando Solar Lezama, Koushik Sen, Ion Stoica - 2024

4 papers in library cite

I like that they worried about contamination. Methodology was nice

Missing author list

[6]Qwen2 Technical Report

2024

3 papers in library cite

Missing author list

[7]Qwen2 Technical Report

2024

3 papers in library cite

[8]Flashattention-2: Faster Attention With Better Parallelism and Work Partitioning

Tri Dao - 2023

2 papers in library cite

Flash attention-2, less important

[9]s1: Simple test-time scaling

Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, Tatsunori Hashimoto - 2025

1 paper in library cites

[10]Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMS Beyond the Base Model?

Y. Yue, Ziru Chen, R. Lu, A. Zhao, Zhengtao Wang, S. Song, G. Huang - 2025

1 paper in library cites

[11]Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models

Qinlang Chen, Lianhui Qin, Joseph Liu, D. Peng, J. Guan, Peng Wang, M. Hu, Y. Zhou, T. Gao, W. Che - 2025

1 paper in library cites

[12]Limo: Less Is More for Reasoning

Yanfang Ye, Zhongqiang Huang, Y. Xiao, E. Chern, S. Xia, P. Liu - 2025

1 paper in library cites

I like the name

[13]Light-R1: Curriculum Sft, Dpo and Rl for Long Cot From Scratch and Beyond

L. Wen, Y. Cai, F. Xiao, Xuan He, Q. An, Z. Duan, Yulun Du, Joseph Liu, T. Tanglifu, X. Lv, Others - 2025

1 paper in library cites

not too cited but Curriculum SFT called my attention

[14]Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models

Siheng Zhao, Z. Xie, Mickel Liu, Jieyun Huang, G. Pang, F. Chen, A. Grover - 2026

1 paper in library cites

[15]Self-Distillation Enables Continual Learning

I. Shenfeld, M. Damani, J. Hubotter, P. Agrawal - 2026

1 paper in library cites

[16]Reinforcement Learning via Self-Distillation

J. Hubotter, F. Lubeck, L. Behric, A. Baumann, M. Bagatella, D. Marta, I. Hakimi, I. Shenfeld, T. K. Buening, C. Guestrin, Others - 2026

1 paper in library cites

[17]Star: Bootstrapping Reasoning With Reasoning

E. Zelikman, Yonghui Wu, J. Mu, N. Goodman - 2022

4 papers in library cite

[18]American invitational mathematics Examination - Aime

Maa - 2024

2 papers in library cite

[19]American invitational mathematics Examination - Aime

Maa - 2024

2 papers in library cite

[20]Learning to Reason With LLMs

Openai - 2024

2 papers in library cite

[21]Deepspeed: System Optimizations Enable Training Deep Learning Models With Over 100 Billion Parameters

J. Rasley, S. Rajbhandari, O. Ruwase, Yun He - 2020

1 paper in library cites

[22]Distilling Reasoning Capabilities Into Smaller Language Models

Kumar Shridhar, A. Stolfo, M. Sachan - 2023

1 paper in library cites

[23]Edge-Grpo: Entropy-Driven Grpo With Guided Error Correction for Advantage Diversity

X. Zhang, S. Wen, Wenhao Wu, L. Huang - 2025

1 paper in library cites

[24]Ghpo: Adaptive Guidance for Stable and Efficient Ilm Reinforcement Learning

Ze Liu, C. Gong, X. Fu, Yibo Liu, R. Chen, S. Hu, S. Zhang, Rosanne Liu, Q. Zhang, D. Tu - 2025

1 paper in library cites

[25]Mind in Society: The Development of Higher Psychological Processes

L. S. Vygotsky - 1978

1 paper in library cites

[26]Olympiadbench: A Challenging Benchmark for Promoting Agi With Olympiad-Level Bilingual Multimodal Scientific Problems

C. He, R. Luo, Yuntao Bai, S. Hu, Z. Thai, Junhong Shen, Jiaxi Hu, X. Han, Y. Huang, Y. Z. Zhang, Others - 2024

1 paper in library cites

[27]Questa: Expanding Reasoning Capacity in LLMS via Question Augmentation

Jeffrey Li, Haowei Lin, H. Lu, K. Wen, Zhilin Yang, Jianfeng Gao, Yonghui Wu, J. Zhang - 2025

1 paper in library cites

[28]Ravr: Reference-Answer-Guided Variational Reasoning for Large Language Models

T. Lin, Xuandong Zhao, X. Zhang, R. Long, Yiheng Xu, Zhejun Jiang, Wenbo Su, Bo Zheng - 2025

1 paper in library cites

[29]Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in Ilm Reasoning

M. Kim, A. Shrestha, S. Shrestha, A. Nepal, K. Ross - 2025

1 paper in library cites

[30]The Role of Tutoring in Problem Solving

D. Wood, J. S. Bruner, G. Ross - 1976

1 paper in library cites

[31]Why Distillation Can Outperform Zero-Rl: The Role of Flexible Reasoning

X. Hu, X. Lu, L. Mao, Y. Z. Zhang, Tong Zhang, B. Wen, Fan Yang, T. Gao, G. Zhou - 2025

1 paper in library cites

[32]Zero: Memory Optimizations Toward Training Trillion Parameter Models

S. Rajbhandari, J. Rasley, O. Ruwase, Yun He - 2020

1 paper in library cites

Cited by

0

papers in your library

Cites

16

papers in your library

Read

on June 17, 2026

Bad methodology and claims are too strong. There is no comprehensive analysis, and lacks comparisons.

Tags

Paper Aliases

No aliases