Skip to main content
A verifiers environment for on-policy trainers. 3 public names. import whileai.simulations as wai, then wai.name.

​
build_tasks

One task per distinct prompt, split into train and holdout. When a prompt has two or more graded rollouts its solve rate is known (partial credit counts as it is) and, with band, prompts the policy always or never solved are dropped: they carry no advantage (rlhfbook.com/c/07-reasoning, difficulty filtering at 20 to 80 percent; DAPO’s dynamic sampling drops accuracy 0 and 1, arXiv:2503.14476). Ungraded prompts and single rollouts are kept as they are. holdout is a fraction, split by scenario id (or the prompt) so a task is wholly on one side, or an explicit list of holdout prompts. Train and holdout are decontaminated against each other at ngram-grams (8: the overlap size rlhfbook.com/c/16-evaluation.html found its contaminations with) and the report says what overlapped.

​
export_environment

Write source as an installable verifiers environment under out. source is a SimulationData (system prompt and tools come from its profile), a row list, or a JSONL path; graded rows get the difficulty band, ungraded rows are exported as they are. reward is a Verifier, a judge callable honoring the SDK judge contract, or 'module:attr'; it must be importable in the trainer process. A @verifier or All([...]) bound to a name in your own module is referenced by that name (a script run as __main__ by its file stem, so keep that directory on the trainer’s path). With no reward the conduct grade is used and the report warns: it is a process reward, and a policy trained on it alone learns to call nothing (see recipes/03-select/prime-intellect-rl). execute names a live world (tool, arguments) -> result; without it the SDK’s mock world answers, seeded per task so every rollout of a task sees the same world. world is a dict of the mock world’s dials (WorldOptions fields: search_hits, exists_share, default_fault_mode, name pools, …); it is written into spec.json and the trainer’s world is built from it, so the world a policy trains against is the one the export says. ngram is the train-vs-holdout decontamination size. Returns the report; the same text is the package README.

​
load_environment

Build the verifiers environment from an exported spec.json. split picks the training task set; the holdout file, when present, becomes eval_dataset. reward and execute override the spec’s references (a callable or 'module:attr'). world (a WorldOptions or a dict of its fields) overrides the mock world’s dials the spec carries; here callables such as fault_modes are fine.