10
20
30
40
50
60
70
80
Rollout step
0.0
0.2
0.4
0.6
0.8
1.0
A Generated trajectory reward
5-step trailing mean
10
20
30
40
50
60
70
80
Rollout step
0%
20%
40%
60%
80%
100%
B Skill-free sampling
5-step trailing mean
Fixed q = 0.20
10
20
30
40
50
60
70
80
Rollout step
0%
20%
40%
60%
80%
100%
C All-wrong groups
5-step trailing mean
10
20
30
40
50
60
70
80
Rollout step
0%
20%
40%
60%
80%
100%
D Zero-variance groups dropped
5-step trailing mean