Pass@k and pass@1 across RL checkpoints
Per-step verifier performance across Snowball and Qwen3-Coder campaigns, followed by within-run change grouped by source dataset.
The gap between pass@1 and pass@k
Each translucent band spans sampled pass@1 to pass@k at a training step (k=16 for Snowball; k=8 for Qwen3-Coder). The selected base-model filter applies to both figures; then filter by source dataset or use the checkboxes. Hover a band to isolate it. Snowball pass@1 uses environment/acc, the outcome-reward series mapped from −1/+1 to 0/1, or the same mapping of binary raw reward, depending on the run. Qwen3-Coder pass@1 is reward/avg_raw_reward for binary-verifier runs: the mean of 512 zero-or-one outcomes. The run using the X0-shaped partial-credit reward has no binary pass@1 and is omitted from this figure.
Within-run change by source dataset
For each included run and metric, each later value is converted to Δ = value(step) − the first recorded value of that metric in the run. Within each source dataset, the bars show the maximum and arithmetic mean across all later-step changes; longer runs contribute more observations. The selected base-model filter applies to both figures. Runs using the DAPO objective with dynamic_sampling.type: filter are excluded because pass@k is conditioned on prompt groups with non-zero reward variance. The run using the X0-shaped partial-credit reward is also excluded because its pass@8 is non-comparable. The entire affected run is removed so pass@k and reward use the same cohort; dataset names do not determine eligibility.
Snowball pass@16 uses reward/avg_pass_at_16; Qwen3-Coder pass@8 uses reward/avg_pass_at_8. Reward uses the corresponding logged raw-reward series. Snowball source datasets come from archived launch records or W&B paths; Qwen source datasets come from the exported model cards and configs. Hover counts report included later-step observations after all exclusions.