Articles tagged with policy-optimization | Vitor Sousa

Vitor Sousa

Articles tagged policy-optimization

Back to blog
After GDPO: A Map of the GRPO Family Part 5 GRPO didn't fragment into a zoo. It exposed a handful of design axes — token vs. sequence granularity, clip width, length and std normalization, KL, advantage normalization, sampling — and every 'new' algorithm is a different point in that space. A practitioner's map of Dr. GRPO, DAPO, GSPO, GDPO, and the frontier, with a decision guide for when to reach for each. ~18 min
GRPO: Eliminating the Value Network Part 3 Group Relative Policy Optimization replaces PPO's learned value function with a simple insight: sample multiple outputs and use their relative rewards as advantages. 33% memory savings, simpler implementation, and the algorithm powering DeepSeek-R1. ~32 min