> ## Documentation Index
> Fetch the complete documentation index at: https://docs.withwhile.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Install with `uv add whileai`; import as `import whileai as wai`.
> Run the offline path first (`simulator=False`, `wai.seeded_agent`, a callable judge); no key is needed for it.
> Report every pass rate with its interval and n, as `scored.pass_at` prints it.

# Endpoint SFT: keep the two ends of the reasoning trace, drop the middle

> Paper: Revisiting Complete Reasoning Traces for Post-Training, Jaehui Hwang et al., arXiv:2609.07103, September 2026.

<Note>The scripts are in the repository, not in the installed package. Clone it,
then `cd recipes/papers/endpoint-sft` before running the commands below. [Browse this recipe on GitHub](https://github.com/whilehq/whileai-sdk/tree/main/recipes/papers/endpoint-sft).</Note>

**Paper:** Revisiting Complete Reasoning Traces for Post-Training, Jaehui Hwang et al., arXiv:2609.07103, September 2026. [https://arxiv.org/abs/2609.07103](https://arxiv.org/abs/2609.07103)
**Book:** in instruction tuning the prompt is masked and "the completions are what the model actually learns from" \[1], so editing the completion text is editing the whole training signal; this recipe changes nothing else.
**Claim:** the middle of a machine-written reasoning trace is weakly attended and carries little that the model cannot re-derive; training on the first and last steps only (about 20% of the tokens removed) matches full-trace SFT and sometimes beats it.
**The change:** the assistant's trace is cut to its first `n` steps plus its last `n` steps before training. Same problems, same model, same learning rate, same optimizer steps, same prompt mask.

## Recipe

1. Base: `Qwen/Qwen2.5-1.5B-Instruct`. Train data: 600 R1 traces from `open-r1/OpenR1-Math-220k`, each one the dataset's own math-verified generation, filtered to 800–3500 tokens and at least 24 steps. Holdout: 64 problems from `HuggingFaceH4/MATH-500`, a different corpus.
2. Baseline arm: TRL `SFTTrainer` + LoRA (r=32), one epoch, lr 1e-4, 8 sequences per optimizer step, 4096-token context, loss on the completion only. The window in step 1 keeps every trace inside that context, so the baseline is never truncated from the right — "full trace" must not quietly mean "the trace minus its answer".
3. Recipe arm: identical, except each trace is cut to its first `n` and last `n` blank-line-separated steps. `n` is one number for the dataset, picked so about 20% of trace tokens go (the paper's §C.3 rule: n=100 on s1K-1.1, n=200 on OpenThoughts3). On this data that is **n = 21**, dropping **19.2%** of trace tokens.
4. Eval: pass\@1 on the same 64 problems, 4 samples each, temperature 0.6, graded by `MathEqual` against the public gold answer. The untrained base is evaluated three times first and that spread is the noise floor; the train set is decontaminated against the holdout before anything trains.
5. Read the Result table with its interval, not its middle number. 64 problems is a small holdout — see Learned.

## Run

```bash theme={"theme":"vitesse-dark"}
python recipe.py --selftest   # the truncation rule, offline: no GPU, no key, no network
python recipe.py --plan       # the same rule on the real data, CPU only: picks n, prints what it drops
python recipe.py              # both arms, budgeted for under 60 GPU minutes on one L40S
python recipe.py --arm recipe --n-holdout 200
```

Needs `MODAL_TOKEN_ID` and `MODAL_TOKEN_SECRET` (Modal workspace `zeroproofai`). `WHILEAI_API_KEY` is optional: with it the run appears on withwhile.com/platform, without it the recipe trains and prints the same numbers.

## Result

Run today, both arms, on one L40S. Round 2 is the headline because round 1's
numbers were set by the eval's token budget rather than by the training: see
Climb.

| Arm                     | pass\@1 | 95% CI        | pass\@k | Steps | GPU min |
| ----------------------- | ------- | ------------- | ------- | ----- | ------- |
| Base, no training       | 0.46    | \[0.36, 0.56] | 0.69    | 0     | 0       |
| Baseline (full trace)   | 0.29    | \[0.21, 0.37] | 0.53    | 75    | 47.7    |
| Recipe (endpoints only) | 0.28    | \[0.20, 0.36] | 0.50    | 75    | 21.7    |

Recipe vs baseline: **-0.012 \[-0.074, +0.047]** over 64 paired problems.
Verdict: **flat**. The interval covers zero, and the proxy check says
**over-optimized**: the shape of the trace moved +0.137 \[+0.070, +0.207] while
pass\@1 did not follow it.

The number that is not in the paper's table: **both SFT arms land below the
untrained base** (0.46 -> 0.29 and 0.28). 600 traces of one epoch on a 1.5B
teaches this model to write like R1 without teaching it to answer like R1. The
paper starts from models that are already trained this way; this recipe starts
from an instruct model, and that difference costs more than the change under
test is worth.

Also measured, on the CPU, before anything trained:

| Measured without a GPU                         | Number                                                    |
| ---------------------------------------------- | --------------------------------------------------------- |
| Train traces after filtering                   | 600, median 50 steps, median 2464 tokens                  |
| `n` chosen by the paper's 20% rule             | 21 steps at each end                                      |
| Trace tokens the cut removes                   | 19.2% (1,464,426 -> 1,181,008 target tokens)              |
| Traces with no middle to remove (\<= 2n steps) | 230 of 600                                                |
| `decontaminate(train, against=holdout)`        | 0 of 600 dropped, and 0 against all 500 MATH-500 problems |

## Checks

Nothing here is ticked by hand: every cell is written by `recipe.py` into `results.json`. These are the round 2 numbers.

| Check                                                            | Source                                       | Result                                                                                                                                                                                                                                                                                                                  |
| ---------------------------------------------------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Eval noise: the base evaluated 3 times, `eval_variance` run\_std | \[2]                                         | **run\_std 0.0135**, so a delta under 0.027 is noise. The measured delta, 0.012, is inside that band twice over                                                                                                                                                                                                         |
| Holdout is clean: `decontaminate(train, against=holdout)`        | \[3]                                         | **run today, CPU: 0 of 600 train rows dropped** (also 0 against all 500 MATH-500 problems). OpenR1-Math-220k comes from NuminaMath and MATH-500 is a slice of the MATH test set, so this was worth measuring rather than assuming                                                                                       |
| Reward is a program, not a judge                                 | \[3]                                         | `MathEqual` against the public MATH-500 gold answer. No judge, no model in the loop                                                                                                                                                                                                                                     |
| Proxy vs target: `delta_report(proxy=)`                          | \[4]                                         | `proxy="marker:trace_form"`: SFT optimizes the *shape* of the trace (a closed `<think>` block ending in `\boxed{}`) whether or not the answer is right. If that rises and pass\@1 does not, the report says over-optimized and the verdict cannot be "moved"                                                            |
| Length: mean completion length before -> after, per arm          | \[4]                                         | **1,619 chars base -> 5,473 baseline, 5,245 recipe.** Both arms more than tripled their output. The recipe arm is trained on shorter targets and still writes nearly as much, so the cut did not buy the brevity it looks like it should                                                                                |
| Hack scan on the last training batch: `hack_scan`                | \[4]                                         | SFT has no per-rollout training reward, so this scans the arm's graded holdout rollouts. Round 1's baseline named **`truncated`** as its top feature — the scan found the eval cap before I did. At 2048 tokens it names ordinary prose (`contains:approach AND contains:two`), which is the scan saying it has nothing |
| Pinned: seed, torch, transformers, trl, peft                     | [the contract](/recipes/papers#the-contract) | seed 17 in the trainer, `--seed 0` for the holdout draw; torch 2.7.1, transformers 4.54.0, trl 0.19.1, peft 0.16.0                                                                                                                                                                                                      |

Both arms share the seed, the problems, the holdout, the grader and every trainer knob. The target text is the only difference, so the delta has one cause available to it.

## Climb

| Round | What changed                                                                                                     | pass\@1                    | vs previous                    |
| ----- | ---------------------------------------------------------------------------------------------------------------- | -------------------------- | ------------------------------ |
| 1     | as the paper: n chosen for a \~20% token drop (n=21 here), 600 traces, 1 epoch, lr 1e-4, eval budget 1024 tokens | baseline 0.16, recipe 0.20 | +0.039 \[-0.016, +0.094], flat |
| 2     | same arms, same seed, eval budget raised 1024 -> 2048 tokens                                                     | baseline 0.29, recipe 0.28 | -0.012 \[-0.074, +0.047], flat |

Round 2 does not change the method; it changes what the eval could see, and
that is why it is the headline. Round 1's `hack_scan` named `truncated` as the
feature separating right answers from wrong ones in the baseline arm: both arms
had learned to write past the 1024-token cap, so the grader was reading
unfinished answers. Doubling the cap recovered 13 points for the baseline and 8
for the recipe — and took the recipe's apparent +0.039 edge with it. That edge
was the recipe arm's shorter output fitting a budget the baseline arm
overflowed, not the cut teaching anything.

The knob the paper says matters is the cutoff itself (its Figure B: flat across
a broad band of retained-step counts, falling off under heavy truncation).
Testing that is round 3, and it is worth doing only on a base that SFT does not
already move backwards.

## Learned

* **The eval budget decided round 1.** At 1024 tokens both trained arms were being graded on answers they had not finished, and the recipe arm looked +0.039 better for writing less. At 2048 the gap is -0.012. Nothing about the method changed between those two numbers. A length-changing intervention measured under a length cap reports the cap.
* **Dropping 19.2% of the trace tokens cost nothing measurable, which is the paper's weaker claim and the one this budget can test.** Its stronger claim is a +1.07 average gain on Qwen3-4B, and a 64-problem holdout at k=4 cannot resolve a point — so read the flat verdict as "no cost detected", not as "no gain exists". The paper's dataset-wide single `n` also leaves 230 of 600 traces byte-identical between the arms, so the change is absent for 38% of the rows before anything trains.
* **Both arms are worse than not training at all** (0.46 -> 0.29 / 0.28), and the proxy check says why: trace *form* moved +0.137 while pass\@1 moved -0.012. The model learned the costume — a closed `<think>` block ending in a boxed answer — and not the reasoning inside it. This is over-optimization \[4] with an SFT loss instead of a reward model, and it is the reason the verdict here could not be "moved" even if the delta had been large.
* Gradient checkpointing is on here, unlike the GRPO recipes next door. Their rule is about trainers that generate while they train, where checkpointing corrupts Qwen generation on these pins. `SFTTrainer` is teacher-forced and never generates, so the rule does not reach it, and 4096-token sequences want the memory back.

Verified 2026-09-17, whileai 0.53, TRL 0.19.1 + PEFT 0.16.0 on torch 2.7.1. 69.4 GPU minutes, $2.31 on one L40S (round 1: 43.1 minutes, $1.42). Run page: [https://withwhile.com/platform/training/run\_aca24d9fdb020c5b](https://withwhile.com/platform/training/run_aca24d9fdb020c5b)

## References

1. Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter *Instruction Tuning*.
2. Lambert, N. Reinforcement Learning from Human Feedback. arXiv:2504.12501, 2025. Chapter *Evaluation*.
3. Lambert, N. et al. Tülu 3: Pushing Frontiers in Open Language Model Post-Training. arXiv:2411.15124, 2024.
4. Gao, L., Schulman, J., Hilton, J. Scaling Laws for Reward Model Overoptimization. ICML 2023. arXiv:2210.10760.