Endpoint SFT: keep the two ends of the reasoning trace, drop the middle

The scripts are in the repository, not in the installed package. Clone it, then cd recipes/papers/endpoint-sft before running the commands below. Browse this recipe on GitHub.

Paper: Revisiting Complete Reasoning Traces for Post-Training, Jaehui Hwang et al., arXiv:2609.07103, September 2026. https://arxiv.org/abs/2609.07103 Book: in instruction tuning the prompt is masked and “the completions are what the model actually learns from” [1], so editing the completion text is editing the whole training signal; this recipe changes nothing else. Claim: the middle of a machine-written reasoning trace is weakly attended and carries little that the model cannot re-derive; training on the first and last steps only (about 20% of the tokens removed) matches full-trace SFT and sometimes beats it. The change: the assistant’s trace is cut to its first n steps plus its last n steps before training. Same problems, same model, same learning rate, same optimizer steps, same prompt mask.

Recipe

Base: Qwen/Qwen2.5-1.5B-Instruct. Train data: 600 R1 traces from open-r1/OpenR1-Math-220k, each one the dataset’s own math-verified generation, filtered to 800–3500 tokens and at least 24 steps. Holdout: 64 problems from HuggingFaceH4/MATH-500, a different corpus.
Baseline arm: TRL SFTTrainer + LoRA (r=32), one epoch, lr 1e-4, 8 sequences per optimizer step, 4096-token context, loss on the completion only. The window in step 1 keeps every trace inside that context, so the baseline is never truncated from the right — “full trace” must not quietly mean “the trace minus its answer”.
Recipe arm: identical, except each trace is cut to its first n and last n blank-line-separated steps. n is one number for the dataset, picked so about 20% of trace tokens go (the paper’s §C.3 rule: n=100 on s1K-1.1, n=200 on OpenThoughts3). On this data that is n = 21, dropping 19.2% of trace tokens.
Eval: pass@1 on the same 64 problems, 4 samples each, temperature 0.6, graded by MathEqual against the public gold answer. The untrained base is evaluated three times first and that spread is the noise floor; the train set is decontaminated against the holdout before anything trains.
Read the Result table with its interval, not its middle number. 64 problems is a small holdout — see Learned.

Run

python recipe.py --selftest   # the truncation rule, offline: no GPU, no key, no network
python recipe.py --plan       # the same rule on the real data, CPU only: picks n, prints what it drops
python recipe.py              # both arms, budgeted for under 60 GPU minutes on one L40S
python recipe.py --arm recipe --n-holdout 200

Needs MODAL_TOKEN_ID and MODAL_TOKEN_SECRET (Modal workspace zeroproofai). WHILEAI_API_KEY is optional: with it the run appears on withwhile.com/platform, without it the recipe trains and prints the same numbers.

Result

Run today, both arms, on one L40S. Round 2 is the headline because round 1’s numbers were set by the eval’s token budget rather than by the training: see Climb.

Arm	pass@1	95% CI	pass@k	Steps	GPU min
Base, no training	0.46	[0.36, 0.56]	0.69	0	0
Baseline (full trace)	0.29	[0.21, 0.37]	0.53	75	47.7
Recipe (endpoints only)	0.28	[0.20, 0.36]	0.50	75	21.7

Recipe vs baseline: -0.012 [-0.074, +0.047] over 64 paired problems. Verdict: flat. The interval covers zero, and the proxy check says over-optimized: the shape of the trace moved +0.137 [+0.070, +0.207] while pass@1 did not follow it. The number that is not in the paper’s table: both SFT arms land below the untrained base (0.46 -> 0.29 and 0.28). 600 traces of one epoch on a 1.5B teaches this model to write like R1 without teaching it to answer like R1. The paper starts from models that are already trained this way; this recipe starts from an instruct model, and that difference costs more than the change under test is worth. Also measured, on the CPU, before anything trained:

Measured without a GPU	Number
Train traces after filtering	600, median 50 steps, median 2464 tokens
`n` chosen by the paper’s 20% rule	21 steps at each end
Trace tokens the cut removes	19.2% (1,464,426 -> 1,181,008 target tokens)
Traces with no middle to remove (<= 2n steps)	230 of 600
`decontaminate(train, against=holdout)`	0 of 600 dropped, and 0 against all 500 MATH-500 problems

Checks

Nothing here is ticked by hand: every cell is written by recipe.py into results.json. These are the round 2 numbers.

Check	Source	Result
Eval noise: the base evaluated 3 times, `eval_variance` run_std	[2]	run_std 0.0135, so a delta under 0.027 is noise. The measured delta, 0.012, is inside that band twice over
Holdout is clean: `decontaminate(train, against=holdout)`	[3]	run today, CPU: 0 of 600 train rows dropped (also 0 against all 500 MATH-500 problems). OpenR1-Math-220k comes from NuminaMath and MATH-500 is a slice of the MATH test set, so this was worth measuring rather than assuming
Reward is a program, not a judge	[3]	`MathEqual` against the public MATH-500 gold answer. No judge, no model in the loop
Proxy vs target: `delta_report(proxy=)`	[4]	`proxy="marker:trace_form"`: SFT optimizes the shape of the trace (a closed `<think>` block ending in `\boxed{}`) whether or not the answer is right. If that rises and pass@1 does not, the report says over-optimized and the verdict cannot be “moved”
Length: mean completion length before -> after, per arm	[4]	1,619 chars base -> 5,473 baseline, 5,245 recipe. Both arms more than tripled their output. The recipe arm is trained on shorter targets and still writes nearly as much, so the cut did not buy the brevity it looks like it should
Hack scan on the last training batch: `hack_scan`	[4]	SFT has no per-rollout training reward, so this scans the arm’s graded holdout rollouts. Round 1’s baseline named `truncated` as its top feature — the scan found the eval cap before I did. At 2048 tokens it names ordinary prose (`contains:approach AND contains:two`), which is the scan saying it has nothing
Pinned: seed, torch, transformers, trl, peft	the contract	seed 17 in the trainer, `--seed 0` for the holdout draw; torch 2.7.1, transformers 4.54.0, trl 0.19.1, peft 0.16.0

Both arms share the seed, the problems, the holdout, the grader and every trainer knob. The target text is the only difference, so the delta has one cause available to it.

Climb

Round	What changed	pass@1	vs previous
1	as the paper: n chosen for a ~20% token drop (n=21 here), 600 traces, 1 epoch, lr 1e-4, eval budget 1024 tokens	baseline 0.16, recipe 0.20	+0.039 [-0.016, +0.094], flat
2	same arms, same seed, eval budget raised 1024 -> 2048 tokens	baseline 0.29, recipe 0.28	-0.012 [-0.074, +0.047], flat

Round 2 does not change the method; it changes what the eval could see, and that is why it is the headline. Round 1’s hack_scan named truncated as the feature separating right answers from wrong ones in the baseline arm: both arms had learned to write past the 1024-token cap, so the grader was reading unfinished answers. Doubling the cap recovered 13 points for the baseline and 8 for the recipe — and took the recipe’s apparent +0.039 edge with it. That edge was the recipe arm’s shorter output fitting a budget the baseline arm overflowed, not the cut teaching anything. The knob the paper says matters is the cutoff itself (its Figure B: flat across a broad band of retained-step counts, falling off under heavy truncation). Testing that is round 3, and it is worth doing only on a base that SFT does not already move backwards.

Learned

The eval budget decided round 1. At 1024 tokens both trained arms were being graded on answers they had not finished, and the recipe arm looked +0.039 better for writing less. At 2048 the gap is -0.012. Nothing about the method changed between those two numbers. A length-changing intervention measured under a length cap reports the cap.
Dropping 19.2% of the trace tokens cost nothing measurable, which is the paper’s weaker claim and the one this budget can test. Its stronger claim is a +1.07 average gain on Qwen3-4B, and a 64-problem holdout at k=4 cannot resolve a point — so read the flat verdict as “no cost detected”, not as “no gain exists”. The paper’s dataset-wide single n also leaves 230 of 600 traces byte-identical between the arms, so the change is absent for 38% of the rows before anything trains.
Both arms are worse than not training at all (0.46 -> 0.29 / 0.28), and the proxy check says why: trace form moved +0.137 while pass@1 moved -0.012. The model learned the costume — a closed <think> block ending in a boxed answer — and not the reasoning inside it. This is over-optimization [4] with an SFT loss instead of a reward model, and it is the reason the verdict here could not be “moved” even if the delta had been large.
Gradient checkpointing is on here, unlike the GRPO recipes next door. Their rule is about trainers that generate while they train, where checkpointing corrupts Qwen generation on these pins. SFTTrainer is teacher-forced and never generates, so the rule does not reach it, and 4096-token sequences want the memory back.

Verified 2026-09-17, whileai 0.53, TRL 0.19.1 + PEFT 0.16.0 on torch 2.7.1. 69.4 GPU minutes,

2.31 on one L40S (round 1: 43.1 minutes,

1.42). Run page: https://withwhile.com/platform/training/run_aca24d9fdb020c5b

References

Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter Instruction Tuning.
Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter Evaluation.
Lambert, N. et al. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv:2411.15124, 2024.
Gao, L., Schulman, J., Hilton, J. Scaling Laws for Reward Model Overoptimization. ICML 2023. arXiv:2210.10760.

Get started

Guides

Concepts

Recipes

Reference

Endpoint SFT: keep the two ends of the reasoning trace, drop the middle

Recipe

Run

Result

Checks

Climb

Learned

References

​Recipe

​Run

​Result

​Checks

​Climb

​Learned

​References

Recipe

Run

Result

Checks

Climb

Learned

References