> ## Documentation Index > Fetch the complete documentation index at: https://docs.withwhile.com/llms.txt > Use this file to discover all available pages before exploring further. > ## Agent Instructions > Install with `uv add whileai`; import as `import whileai as wai`. > Run the offline path first (`simulator=False`, `wai.seeded_agent`, a callable judge); no key is needed for it. > Report every pass rate with its interval and n, as `scored.pass_at` prints it. # Two hosted runs on the same rows, and the warning only one of them gives you > Seat: an open-model fine-tuner who lives in TRL and Hugging Face, here for clean exports, a LoRA run, a hosted result and a cost line. The scripts are in the repository, not in the installed package. Clone it, then `cd recipes/community/hosted-grpo-vs-sft` before running the commands below. [Browse this recipe on GitHub](https://github.com/whilehq/whileai-sdk/tree/main/recipes/community/hosted-grpo-vs-sft). **Seat:** an open-model fine-tuner who lives in TRL and Hugging Face, here for clean exports, a LoRA run, a hosted result and a cost line. **Question:** the customer-simulation ledger ([#31](https://github.com/whilehq/whileai-sdk/issues/31)) has said four times that nobody has run `method="grpo"` on the **hosted** path. So: run hosted SFT and hosted GRPO on *the same 84 rows*, measure both against the same base on a pinned holdout, and see whether the adapter can be pulled back out into PEFT form. **The answer, and it is not about GRPO.** Neither method moved the target on 19 tasks (nowhere near enough tasks — `push_rows` said so before I trained). What the run actually found is that the **hosted trainer silently disagrees with itself about which rows to train on**, in two different directions, and neither one tells you: | | rows in the pushed set | rows the run reports using | what I expected | | --------------- | ---------------------- | -------------------------- | -------------------------------------------------- | | `method="sft"` | 84 (12 pass, 72 fail) | **84** | 12 — the docstring says "LoRA on the passing rows" | | `method="grpo"` | 84 (over 21 tasks) | **6** | 84 — nothing said otherwise | And the SDK *already knows* this is dangerous. The local exporter, on the identical rows, refuses to stay quiet: ```text theme={"theme":"vitesse-dark"} 72 of 84 rows have reward below 0.5 and are exported as SFT targets; a model trained on them learns the failure. Pass `scored.passes()` (or filter on reward) unless that is intended. ``` `wai.export_training` prints that. `wai.train(method="sft")` does not, and then trains on those 72 rows. **The warning predicted my result:** the SFT adapter half-stopped using its tools at all, `called_lookup` **0.987 → 0.487, −0.500 \[−0.697, −0.303], p=0.0025**. It learned the failure, exactly as the other half of the product said it would. What you will learn: what the hosted `sft` / `grpo` / `dpo` methods actually consume, why `profile()["mixed_tasks"]` is the number that predicts whether a GRPO run will do anything, why a guard marker belongs in `must_not_regress` even when it reads 1.000, and that `wai.hf_publish_run` works in 20 seconds. You need `WHILEAI_API_KEY`; `--dry-run` needs nothing. Two hosted runs, **under \$1** of GPU (see **Cost**, and note the SDK gave me no way to compute that). ## Run it ```bash theme={"theme":"vitesse-dark"} uv add whileai # 0.75 cd recipes/community/hosted-grpo-vs-sft python run.py # data -> train -> eval -> report python run.py --dry-run # offline: the rule, on hand-built traces. No key, no GPU python run.py report # re-print from saved rows, offline python run.py export # hf_publish_run on the trained adapter ``` | flag | default | what it does | | ----------- | ------- | ------------------------------------------------------------- | | `step` | `all` | `data`, `train`, `eval`, `report`, `export` | | `--budget` | 160 | rollouts the offline template writer draws the task grid from | | `--repeats` | 4 | rollouts per task, so the paired test means something | | `--holdout` | 0.35 | fraction held out by `split_pseudo_production` | | `--steps` | 20 | GRPO optimizer steps | | `--epochs` | 2.0 | SFT epochs | | `--dry-run` | off | offline, no key, no GPU | ## The rule One deterministic marker over **tool calls only** — no prose is parsed, no dollar figure is matched. Three separate ledger entries found their headline was a defect in a grader that read the reply text, so this one does not read reply text at all. | marker | meaning | role | | ------------------------------------------------------ | -------------------------------------------- | ---------------------------- | | `resolved_with_lookup` | looked the order up **and then** refunded it | target | | `no_blind_refund` | never refunded an id it had not looked up | guard, in `must_not_regress` | | `called_lookup`, `attempted_refund`, `called_any_tool` | the halves, separately | diagnosis | The guard reads **1.000 on every arm** and is still worth carrying: it is satisfiable by inaction, so it can only ever regress, which is precisely what a guard is for. The target is the one with headroom (base 0.250). `python run.py --dry-run` pins all six rows of this table offline, including the two degenerate cases ("asked for the id" and "lookup only" both fail the target). **Denominator, stated plainly.** The task grid contains prompts that legitimately have no order id, where refusing to refund is correct. They sit in the denominator, so the *level* of `resolved_with_lookup` is not an accuracy. The task set is pinned and identical across all three arms, so the paired *delta* is like for like — which is all this recipe claims. ## What I actually ran * 160 rollouts from `wai.simulate` (offline template writer, no model key), 40 prompts × 4 repeats, agent = served `qwen3-4b-think` through `wai.local_model`, 108 s * `split_pseudo_production(fraction=0.35)` → 84 train / 76 holdout * Pushed the **same 84 train rows twice**, under two ids. `wai.train` on a dataset that is already training answers with *that* run instead of starting a second one, so two methods over identical rows need two dataset ids. * Both runs: `base_model="Qwen/Qwen3-4B"`, `lora_r=16`, `seed=11`, same holdout set * All three arms evaluated through **one entry point** (`wai.local_model`, `thinking=False`, `temperature=0.8`), on the same 76 pinned holdout rows / 19 tasks, k=4 — the same-entry-point discipline from [`same-entrypoint-before-after`](/recipes/community/same-entrypoint-before-after) ## Results **The power warning came first, unprompted, from `push_rows`:** ```text theme={"theme":"vitesse-dark"} holdout has 19 tasks at k=4; proving a 5% gain at 80% power needs about 352 (holdout_size). A smaller holdout reads a real gain that size as no_change_detected. ``` Believe that before you read anything below. 19 tasks resolves nothing subtle. | arm | n | `resolved_with_lookup` | `no_blind_refund` | `called_lookup` | | --------------------- | -- | ---------------------- | ----------------- | --------------- | | base `qwen3-4b-think` | 76 | 0.250 | 1.000 | 0.987 | | hosted **SFT** | 76 | 0.145 | 1.000 | **0.487** | | hosted **GRPO** | 76 | 0.303 | 1.000 | 1.000 | Paired, by task, 95% bootstrap interval from `wai.delta_report`, n\_paired = 19: | base → | metric | delta \[95%] | p | verdict | | -------- | ---------------------- | ---------------------------- | ---------- | ------------------------ | | **SFT** | `resolved_with_lookup` | −0.105 \[−0.250, +0.039] | 0.298 | `no_change_detected` | | **SFT** | `called_lookup` | **−0.500 \[−0.697, −0.303]** | **0.0025** | `a_better` (base better) | | **SFT** | `no_blind_refund` | +0.000 \[+0.000, +0.000] | 1.000 | guard held | | **GRPO** | `resolved_with_lookup` | +0.053 \[−0.013, +0.158] | 0.496 | `no_change_detected` | | **GRPO** | `called_lookup` | +0.013 \[+0.000, +0.039] | 1.000 | `no_change_detected` | | **GRPO** | `no_blind_refund` | +0.000 \[+0.000, +0.000] | 1.000 | guard held | `delta_report` put both tool markers in `slipped` for the SFT arm and left `regressions` empty, since the *target* did not regress. The interesting number is in `slipped`, not in the verdict — worth knowing before you read only the headline. **Neither adapter beat the base on the target.** Say that plainly. The SFT arm is a measured, significant behavioral regression on tool use; the GRPO arm is a +0.053 that 19 tasks cannot resolve from zero. ### Why GRPO did nothing: 6 rows ```text theme={"theme":"vitesse-dark"} summary.train = {"rows": 84, "used": 6} # and holdout {"rows": 76, "used": 8} last = {"reward": 0.10, "reward_std": 0.0, "grad_norm": 0.0} ``` GRPO needs a group with both a pass and a fail for the advantage to be non-zero, and `wai.profile(dataset_id)` had already computed exactly how many of my tasks qualified: ```text theme={"theme":"vitesse-dark"} pass_rate 0.143 · split {pass 12, fail 72} · tasks 21 · mixed_tasks 6 · support 0.238 ``` **`mixed_tasks: 6` is the number that predicted this run would be a no-op**, and it is available before you spend the GPU. Nothing in `wai.train` looked at it or warned. 20 steps over 6 prompts ran 3.3 epochs and ended at `reward_std 0.0`, `grad_norm 0.0` — a collapsed run that still reports `status: done`. The hosted GRPO reward is fixed (`config.reward = "reference first action"`, `config.gold = "judge"`); you do not pass a reward function on this path. That is a real design choice and it is not documented at the call site. The run's hack monitor did fire, on its own: ```text theme={"theme":"vitesse-dark"} step 10 · regime reward_hack · top_feature "contains:. first AND contains:it" · rho_max 0.6983 ``` On a run whose reward never moved, I read that as the monitor finding structure in noise rather than a real exploit — but it cost nothing and it is the right instinct. ### The export half: this worked ```python theme={"theme":"vitesse-dark"} wai.hf_publish_run("run_7d45f7e7f724464a", repo="refund-lookup-grpo-qwen3-4b", private=True) ``` 20 seconds, 148 MB, 10 files, private repo, commit tagged `zp-run_7d45f7e7f724464a`: [`jaweiss2305/refund-lookup-grpo-qwen3-4b`](https://huggingface.co/jaweiss2305/refund-lookup-grpo-qwen3-4b). `wai.hf_status()` told me the account was connected and which namespaces I could write to *before* I spent anything, which is the check I wanted and did not expect to find. I did not verify the repo loads under `peft.PeftModel.from_pretrained` — the repo is private and this sandbox has no `HF_TOKEN`. 148 MB is larger than an r=16 LoRA on a 4B needs, so check what the 10 files are before assuming it is adapter-only. `wai.export_training(rows, path, format=...)` is the local half and is good: OpenAI wire format by default (`arguments` as a JSON string), `format="trl"` for the dicts a chat-template trainer wants, `mask_mode`, `strip_think`, and the reward warning quoted at the top of this README. ## Cost | item | measured | my arithmetic | | --------------------------------------------------- | ---------------------------------- | ------------- | | hosted SFT | **56.4 s** on A10G | \~\$0.02 | | hosted GRPO | **329.5 s** on L40S | \~\$0.18 | | 5 rollout passes (data + 3 eval arms) + cold starts | \~10 min on the shared serving GPU | \~\$0.2–0.4 | | `hf_publish_run` | 20 s | — | | **total** | | **under \$1** | **The dollar column is mine, not the product's.** Six ledger entries have now asked for a cost number. `run.summary` gives `seconds` and `gpu` — which is genuinely more than it used to — but there is no published rate anywhere in the SDK, `wai.models()`, or withwhile.com, so the right-hand column is me multiplying by a GPU price I brought from outside. Do not cite it as a While number. ## What I would do next 1. **Size the set before training, not after.** `profile()` gives `mixed_tasks` and `support`; a GRPO run wants those to be large. My `support: 0.238` was the tell. 2. **Filter the SFT set yourself.** Until `train(method="sft")` matches its docstring, push only passing rows (`scored.passes()`, or filter `reward == 1.0`) into the set you hand it. My 72 failure rows went straight into the adapter. 3. **Get more tasks.** 19 is not an experiment. `holdout_size` says 352 for a 5% gain; the offline template writer will produce them and they cost no model key. 4. Re-run GRPO on a set built *for* it — tasks deliberately sampled to be mixed — and see whether the hosted path can move anything at all. That question is still open: this run did not test GRPO, it tested a 6-row GRPO run. ## Issues this run filed * [#396](https://github.com/whilehq/whileai-sdk/issues/396) `train(method="sft")` trains on failing rows; `export_training` warns and `train` does not * [#397](https://github.com/whilehq/whileai-sdk/issues/397) `train(method="grpo")` silently used 6 of 84 rows * [#398](https://github.com/whilehq/whileai-sdk/issues/398) `dataset_report("ds_...")` returns an all-zero report instead of raising * [#399](https://github.com/whilehq/whileai-sdk/issues/399) no cost number anywhere * [#400](https://github.com/whilehq/whileai-sdk/issues/400) `simulate()` prints nothing until it returns ## Run ids | what | id | | ------------------ | -------------------------------------------------------------------------------------- | | hosted SFT | [`run_80323eb0f1f1d19e`](https://withwhile.com/platform/training/run_80323eb0f1f1d19e) | | hosted GRPO | [`run_7d45f7e7f724464a`](https://withwhile.com/platform/training/run_7d45f7e7f724464a) | | train set (sft) | `ds_68c3bac292fe6222` | | train set (grpo) | `ds_25d05bb486e0aaae` | | holdout | `ds_a52518f20192429c` | | adapter on the Hub | `jaweiss2305/refund-lookup-grpo-qwen3-4b`, tag `zp-run_7d45f7e7f724464a` | The three datasets were deleted after the run (scratch); the runs and the two served models are kept, and the serving endpoint scales to zero on its own.