> ## Documentation Index > Fetch the complete documentation index at: https://docs.withwhile.com/llms.txt > Use this file to discover all available pages before exploring further. # whileai.simulations.ingest > Platform calls: datasets, traces, cuts, training runs, hosted models. Platform calls: datasets, traces, cuts, training runs, hosted models. 39 public names. `import whileai.simulations as wai`, then `wai.name`. | Name | What it does | | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [`agents`](#agents) | Every agent on your account with counts: traces, sets by purpose, public cards. | | [`catalog`](#catalog) | The public catalog: `{"datasets": [card, ...], "agents": [...]}`. | | [`cut`](#cut) | Cut training data out of your traces: the platform's "Make training data" button, as one line. | | [`cuts`](#cuts) | How your traces group into prompts, and what a cut would hold. | | [`datasets`](#datasets) | List your datasets plus storage used, newest first. | | [`delete_dataset`](#delete_dataset) | Permanently delete a dataset from your account. | | [`delete_empty_datasets`](#delete_empty_datasets) | Delete datasets with no stored bytes, or with `max_rows` rows or fewer when that is set (smoke runs). | | [`dimensions_from_traces`](#dimensions_from_traces) | Coverage axes aimed at behaviors seen in `rows`. | | [`drop_leaky_rows`](#drop_leaky_rows) | Kept rows plus the report. | | [`flaw_rows`](#flaw_rows) | Rows with an observed fault or a 0 label: the next round's traces. | | [`format_cuts`](#format_cuts) | What `cuts()` says, as the sentence the traces page leads with. | | [`format_trace_report`](#format_trace_report) | The trace report as a text block, aiming stated in plain words. | | [`hf_publish`](#hf_publish) | Push one of your datasets to a Hugging Face dataset repo you own. | | [`hf_publish_run`](#hf_publish_run) | Push a finished training run's LoRA adapter to a Hugging Face model repo you own, with a model card (base model, metrics, the dataset repo when the data was pushed too). | | [`hf_status`](#hf_status) | Is a Hugging Face account connected to this account, and which namespaces (you plus your orgs) can it publish under? | | [`import_hf`](#import_hf) | Bring one split of a Hugging Face dataset onto your account as rows, so it gets a profile (pass rate, gradient support, mixed prompts) before you train on it. | | [`infer_harness`](#infer_harness) | Draft a harness from observed trace rows. | | [`issue_delegated_credential`](#issue_delegated_credential) | Create a short-lived delegated credential for SDK or backend use. | | [`leakage_report`](#leakage_report) | Near-copy check of generated prompts against source traces. | | [`load_traces`](#load_traces) | Normalize any supported trace source to the canonical schema above. | | [`mine_traces`](#mine_traces) | What the deployed agent actually did, counted for grid focusing. | | [`opening_share`](#opening_share) | Share of traces whose conversation opens with the assistant. | | [`preview`](#preview) | Three sample rows and the analyzer report for one of your datasets. | | [`profile`](#profile) | The trainer's numbers for one of your datasets: pass rate, gradient support, tasks with both a pass and a fail, tool use, tokens, per-task pass rates. | | [`publish`](#publish) | Publish one of your datasets as a public card on zeroproofai.com/datasets. | | [`pull`](#pull) | Download a dataset. | | [`purge_agent`](#purge_agent) | Remove an agent and everything under it: its traces, its datasets, and its registry record. | | [`push_file`](#push_file) | Upload an existing JSONL file. | | [`push_rows`](#push_rows) | Upload rows as JSONL to your While account. | | [`refresh_delegated_credential`](#refresh_delegated_credential) | Refresh a delegated credential before it expires. | | [`register_agent`](#register_agent) | Create or update an agent record: the name, and optionally what it is (a line), its tool schemas, and its system prompt. | | [`rows_from_otel`](#rows_from_otel) | Conversations from an OTLP JSON batch or an iterable of spans. | | [`send_score`](#send_score) | Grade a run that has already finished — the number `cut()` filters on:: | | [`simulate_from_traces`](#simulate_from_traces) | Alias for `simulate(agent, traces=...)`: same grid focus and leakage gate, for callers who start from the traces. | | [`split_pseudo_production`](#split_pseudo_production) | Set aside a pseudo-production slice; the rest stays for training. | | [`tools_from_traces`](#tools_from_traces) | The agent's tool surface, read off the calls the traces contain. | | [`trace_report`](#trace_report) | What these traces contain and what they will aim generation at. | | [`unpublish`](#unpublish) | Take a dataset off the public catalog. | | [`update_dataset`](#update_dataset) | Change what a dataset is for, its mode, agent or description. | ## otel OTel GenAI spans to trajectory rows: `simulate(traces=...)` input. ### rows\_from\_otel ```python theme={null} rows_from_otel( source: Any, reward_keys: Sequence[str] = ('whileai.reward',), ) -> list[dict] ``` Conversations from an OTLP JSON batch or an iterable of spans. One row per conversation id (trace id when no conversation id is emitted): the first user message becomes `prompt`, tool spans in time order become `steps`, later user turns become user steps, and the last assistant output becomes `final_text`. `reward_keys` names the span attributes a 0..1 reward is read from (`REWARD_KEYS`). ## platform While platform client: dataset upload, listing, download. ### agents ```python theme={null} agents(api_key: str | None = None) -> list[dict] ``` Every agent on your account with counts: traces, sets by purpose, public cards. An agent exists the moment a push names it (`data.push(name, agent=...)`) or a trace arrives with `gen_ai.agent.name`; `register_agent` is for attaching the spec or a description ahead of that. ### catalog ```python theme={null} catalog() -> dict ``` The public catalog: `{"datasets": [card, ...], "agents": [...]}`. No key needed. ### cut ```python theme={null} cut( agent: str | None = None, kind: str = 'rl', since: str = 'all', holdout: float | None = None, band: tuple[float, float] | None = None, name: str | None = None, api_key: str | None = None, **filters: Any, ) -> dict ``` Cut training data out of your traces: the platform's "Make training data" button, as one line. Runs of the same prompt are grouped by `zeroproof.scenario_id`. `kind="rl"` keeps the prompts the agent passes some of the time and not always (20% to 80% by default, RLHF book ch. 7); `kind="sft"` keeps the best run of every prompt that ever passed. Either way the prompts are split into a train set and a held-out set, so a prompt you measure on is never a prompt you trained on. `since` is the trace window (`"all"`, `"24h"`, `"7d"`); any other keyword is a trace filter, the same ones the traces page has (`model=`, `tool=`, `evalSet=`, `measure=`). `band=(lo, hi)` moves the pass-rate window and `holdout=` the split fraction. Returns the gate's reply with the two sets pulled out:: made = wai.cut(agent="my-agent", kind="rl") wai.pull(made\["train"]\["datasetId"], "train.jsonl") made\["holdout"]\["datasetId"] # measure on this, never train on it ### cuts ```python theme={null} cuts( agent: str | None = None, since: str = 'all', api_key: str | None = None, **filters: Any, ) -> dict ``` How your traces group into prompts, and what a cut would hold. The numbers the platform's traces page shows above the runs table: `prompts` seen, `rl` (the ones worth training on, with the train and holdout counts), `sft`, and `more` (prompts that need more runs). Read it before `cut()` when you want to know what you would get. `format_cuts()` prints it as the three lines the traces page shows. ### datasets ```python theme={null} datasets(api_key: str | None = None) -> dict ``` List your datasets plus storage used, newest first. ### delete\_dataset ```python theme={null} delete_dataset(dataset_id: str, api_key: str | None = None) -> dict ``` Permanently delete a dataset from your account. ### delete\_empty\_datasets ```python theme={null} delete_empty_datasets( max_rows: int = 0, dry_run: bool = False, api_key: str | None = None, ) -> dict ``` Delete datasets with no stored bytes, or with `max_rows` rows or fewer when that is set (smoke runs). Permanent. Returns the ids. ### format\_cuts ```python theme={null} format_cuts(report: Mapping[str, Any], agent: str | None = None) -> str ``` What `cuts()` says, as the sentence the traces page leads with. The answer, the counts behind it, and the line to run next:: 3 prompts are worth training on 9 runs · 3 train · none held out next: wai.cut(agent="my-agent", kind="rl") ### hf\_publish ```python theme={null} hf_publish( dataset_id: str, namespace: str | None = None, repo: str | None = None, private: bool = False, wait: bool = True, timeout: float = 600.0, poll: float = 2.5, api_key: str | None = None, ) -> dict ``` Push one of your datasets to a Hugging Face dataset repo you own. The set's purpose (train, holdout, eval) is the split; pushing the holdout set into the same `repo` adds a second split. Every push is a commit tagged `zp-` and the repo carries `whileai.json` (split -> dataset, numbers, history). Defaults: your username and a slug of the dataset name. With `wait` (the default) this returns the finished state `{"repo", "url", "commit", "tag", "split", ...}`; otherwise the `pushing` stamp. ### hf\_publish\_run ```python theme={null} hf_publish_run( run_id: str, namespace: str | None = None, repo: str | None = None, private: bool = True, wait: bool = True, timeout: float = 900.0, poll: float = 3.0, api_key: str | None = None, ) -> dict ``` Push a finished training run's LoRA adapter to a Hugging Face model repo you own, with a model card (base model, metrics, the dataset repo when the data was pushed too). Private by default: it is a checkpoint, not a release. Tagged `zp-`. ### hf\_status ```python theme={null} hf_status(api_key: str | None = None) -> dict ``` Is a Hugging Face account connected to this account, and which namespaces (you plus your orgs) can it publish under? Connect one on any dataset page at zeroproofai.com/platform/datasets. Returns `{"connected", "username", "namespaces", "scopes"}`. ### import\_hf ```python theme={null} import_hf( repo: str, split: str = 'train', revision: str | None = None, config: str | None = None, name: str | None = None, purpose: str = 'train', mode: str | None = None, agent: str | None = None, description: str | None = None, max_rows: int = 100000, wait: bool = True, timeout: float = 900.0, poll: float = 3.0, api_key: str | None = None, ) -> dict ``` Bring one split of a Hugging Face dataset onto your account as rows, so it gets a profile (pass rate, gradient support, mixed prompts) before you train on it. Parquet, JSONL, CSV and Arrow all come in the same way. Public repos need no connected account; private ones use the Hugging Face account connected on the platform. Returns the dataset row. With `wait` (the default) the row is `ready` (or this raises with the import error); otherwise it is `importing` and `wai.datasets()` shows it settle. ### issue\_delegated\_credential ```python theme={null} issue_delegated_credential( clerk_token: str | None, ttl_seconds: int = 3600, name: str = 'sdk-default', timeout: float = 120, ) -> dict ``` Create a short-lived delegated credential for SDK or backend use. `clerk_token` must be a valid Clerk session token or other authenticated backend token. This helper sends that token as a bearer token to the auth endpoint to mint the delegated credential. ### preview ```python theme={null} preview(dataset_id: str, api_key: str | None = None) -> dict ``` Three sample rows and the analyzer report for one of your datasets. ### profile ```python theme={null} profile( dataset_id: str, force: bool = False, api_key: str | None = None, ) -> dict ``` The trainer's numbers for one of your datasets: pass rate, gradient support, tasks with both a pass and a fail, tool use, tokens, per-task pass rates. Cached on the platform until the set changes; `force=True` recomputes. ### publish ```python theme={null} publish( dataset_id: str, agent: str, description: str | None = None, api_key: str | None = None, ) -> dict ``` Publish one of your datasets as a public card on zeroproofai.com/datasets. Cards are grouped by `agent` (a short name such as `"airline-support"`). The dataset must be finalized and hold rows. Returns the card. Anyone can then `pull` it with no key. ### pull ```python theme={null} pull( dataset_id: str, path: str | None = None, api_key: str | None = None, ) -> str | list[dict] ``` Download a dataset. Writes JSONL to `path` and returns the path, or returns the parsed rows when `path` is omitted. Public catalog datasets need no key; your own need the usual one. A dataset is stored as one or more parts, and the grant lists every one of them. Datasets pushed with `push_rows` are a single part, which is why reading only `downloadUrl` looked correct for so long; a dataset filled by trace ingest is one part per trace, and that path returned the first row of a 60-row dataset without saying so. ### purge\_agent ```python theme={null} purge_agent( agent: str, dry_run: bool = False, api_key: str | None = None, ) -> dict ``` Remove an agent and everything under it: its traces, its datasets, and its registry record. Permanent. `dry_run=True` only counts. Returns `{"agent", "traces", "datasets", "deleted"}`. ### push\_file ```python theme={null} push_file( path: str, name: str | None = None, api_key: str | None = None, parent: str | None = None, gate: bool = True, mode: str | None = None, ) -> dict ``` Upload an existing JSONL file. `name` defaults to the file name. `gate=True` (default) parses the file, runs `publish_gate` (rows get their `calibration` stamp; RL-shaped rows that are ungraded or have no mixed group are refused), and uploads the stamped rows. The report comes back as `entry["gate"]`. `gate=False` uploads the bytes exactly as they are on disk. ### push\_rows ```python theme={null} push_rows( rows: list[dict], name: str, api_key: str | None = None, parent: str | None = None, gate: bool = False, mode: str | None = None, purpose: str | None = None, agent: str | None = None, description: str | None = None, endorsed: Sequence[str] = (), strict_hacks: bool = False, prove_effect: float = 0.05, ) -> dict ``` Upload rows as JSONL to your While account. Returns the registry entry, including `datasetId`. Pass `parent` (a `ds_...` id) when this dataset is an iteration of an existing one, so lineage shows on the platform. `purpose` is what the set is for on the Datasets page: `"train"` (the default), `"holdout"` or `"eval"`; `mode` is the simulation mode that made it, and is also recorded. `gate=True` runs `publish_gate` first (calibration stamp; RL-shaped rows refused when ungraded or without a mixed group) and returns its report as `entry["gate"]`. `endorsed` names what the reward should track for the gate's `hack_scan`; `strict_hacks=True` refuses a set whose reward is best explained by something else. `SimulationData.push` gates by default; this row-level entry point does not, because the caller may already have run `optimize`. A `purpose="holdout"` push warns when the set is too small to prove a `prove_effect` gain (5 points) at 80% power. ### refresh\_delegated\_credential ```python theme={null} refresh_delegated_credential( clerk_token: str | None, credential: str, ttl_seconds: int = 3600, timeout: float = 120, ) -> dict ``` Refresh a delegated credential before it expires. ### register\_agent ```python theme={null} register_agent( name: str, description: str | None = None, tools: list | None = None, system_prompt: str | None = None, api_key: str | None = None, ) -> dict ``` Create or update an agent record: the name, and optionally what it is (a line), its tool schemas, and its system prompt. Returns the record. ### send\_score ```python theme={null} send_score( trace_id: str | None = None, value: float | bool | None = None, name: str = 'score', scores: Sequence[dict] | None = None, api_key: str | None = None, **fields: Any, ) -> dict ``` Grade a run that has already finished — the number `cut()` filters on:: wai.send\_score("4bf92f3577b34da6", 1.0) # passed wai.send\_score("4bf92f3577b34da6", 0.0) # failed wai.cut(agent="my-agent", kind="rl") # now there is something to cut A run counts as a pass at **1.0 or above**, which is what puts its prompt in the 20-80% band `cut(kind="rl")` keeps. Send a 0-to-1 quality number as the verdict and nothing will ever read as a pass; scale it, or send it under its own `name=` and leave `score` for the verdict. This is a call of its own because a judge answers after the run it is judging has closed, and a human disagreeing with the judge answers a day later. Re-sending the same `name` is a correction, not a duplicate:: wai.send\_score("4bf92f3577b34da6", 0.82, name="helpfulness") wai.send\_score(scores=\[\{"traceId": t, "value": v} for t, v in graded]) Any other keyword rides along to every measurement sent: `label=`, `pass_at=`, `max=`, `description=`, `direction=`, `kind=`. Returns the gate's reply — `applied`, plus `unknown` (trace ids this account never sent) and `rejected` when a batch is partly good. Raises when nothing landed at all, because a typo in a trace id is otherwise a silent success. ### unpublish ```python theme={null} unpublish(dataset_id: str, api_key: str | None = None) -> dict ``` Take a dataset off the public catalog. The data stays on your account. ### update\_dataset ```python theme={null} update_dataset( dataset_id: str, purpose: str | None = None, mode: str | None = None, agent: str | None = None, description: str | None = None, api_key: str | None = None, ) -> dict ``` Change what a dataset is for, its mode, agent or description. `purpose` moves it between the Train, Holdout, Eval and Raw sections of the Training data page. Only the arguments you pass change. ## traces Production traces to a focused coverage grid. Simulation, version two. ### dimensions\_from\_traces ```python theme={null} dimensions_from_traces( rows: Sequence[dict], tools: list[dict], policy: str = '', broaden: bool = True, fault_to_axis: Mapping[str, tuple[str, str]] | None = None, ) -> dict[str, list[str]] ``` Coverage axes aimed at behaviors seen in `rows`. Starts from `build_dimensions` for this agent so every value is one the writer and sandbox understand. The tool axis puts observed failing tools first; `broaden=False` drops tools the traces never touched (keeping the base specials such as `unrelated`), so a run spends its budget near the flaws instead of boiling the ocean. Fault and world axes always keep their clean value: contrast needs passing rows too. `fault_to_axis` maps an observed fault chip to the axis value that reproduces it (`FAULT_TO_AXIS` by default). ### drop\_leaky\_rows ```python theme={null} drop_leaky_rows( rows: Sequence[dict], sources: Sequence[Any], threshold: float = 0.9, embedder: Any = 'hash', examples: int = 20, ) -> tuple[list[dict], dict[str, Any]] ``` Kept rows plus the report. Flagged rows are removed, not rewritten. ### flaw\_rows ```python theme={null} flaw_rows(rows: Sequence[dict]) -> list[dict] ``` Rows with an observed fault or a 0 label: the next round's traces. The hill-climb loop feeds a round's failures back into `simulate(traces=flaw_rows(evaluated))` so the next batch aims at what the agent still gets wrong. ### format\_trace\_report ```python theme={null} format_trace_report(report: dict[str, Any]) -> str ``` The trace report as a text block, aiming stated in plain words. ### infer\_harness ```python theme={null} infer_harness(rows: Sequence[dict]) -> dict[str, Any] ``` Draft a harness from observed trace rows. Mechanical, no model. Tool schemas come from what the agent actually sent: every argument key seen for a tool becomes a property, its JSON type read off the observed values, and a key present on every call becomes required. The policy cannot be inferred — exporters do not ship system prompts — so it comes back empty for the caller to fill in. A drafted schema is a starting point to edit, not a spec to trust: it can only describe arguments the traces happened to exercise. ### leakage\_report ```python theme={null} leakage_report( generated: Sequence[Any], sources: Sequence[Any], threshold: float = 0.9, embedder: Any = 'hash', examples: int = 20, ) -> dict[str, Any] ``` Near-copy check of generated prompts against source traces. A generated row whose prompt sits at or above `threshold` cosine similarity to any source prompt is flagged (0.9 by default: the 8-gram exact-overlap test of rlhfbook.com/c/16-evaluation.html with a small paraphrase allowance). Exact matches always flag, whatever the embedder thinks. `leaky` lists the first `examples` offenders; `n_leaky` is the full count. ### load\_traces ```python theme={null} load_traces(source) -> list[dict] ``` Normalize any supported trace source to the canonical schema above. `source` is a JSONL path or an iterable of dicts. Rows carrying `tool_trace`/`trace` instead of `steps`, `final`/`output`/ `response` instead of `final_text`, or only OpenAI-style `messages` are converted; `reward` is kept only when it coerces cleanly to 0 or 1, and its absence is fine. Rows that are not dicts or carry neither an ask nor any steps are dropped. ### mine\_traces ```python theme={null} mine_traces(rows: Sequence[dict]) -> dict[str, Any] ``` What the deployed agent actually did, counted for grid focusing. `flaw_rows` is any row with an observed fault or a 0 label. Those are the behaviors worth simulating more of. ### opening\_share ```python theme={null} opening_share(rows: Sequence[dict]) -> float ``` Share of traces whose conversation opens with the assistant. Reads the raw `messages` field (kept by `load_traces`); rows without messages count as user-opened. This is the evidence the `opening="auto"` topology axis resolves against. ### simulate\_from\_traces ```python theme={null} simulate_from_traces( traces: Sequence[dict], agent: Any = None, tools: list[dict] | None = None, policy: str = '', mode: str = 'rl', **kwargs: Any, ) ``` Alias for `simulate(agent, traces=...)`: same grid focus and leakage gate, for callers who start from the traces. With no agent, tools or policy, the tool surface is read from the traces themselves, so handing over graded telemetry is enough to start. ### split\_pseudo\_production ```python theme={null} split_pseudo_production( rows: Sequence[dict], fraction: float = 0.2, seed: int = 0, ) -> tuple[list[dict], list[dict]] ``` Set aside a pseudo-production slice; the rest stays for training. The split is by task, not by row: every row sharing a `task_key` (the `scenario_id`, else the prompt) lands on the same side, so the held-out slice is disjoint from the training side in the unit every report groups by, not just prompt-disjoint. Splitting by row is not enough — under `mode="rl"` with `repeats=k` each prompt has k rows, and scattering siblings across the two sides trains the student on every prompt it is then evaluated on. Every unique flaw signature (fault name plus behavior shape) sends its task to the production side first, so the held-out slice contains each distinct failure at least once. `fraction` is still counted in rows, but whole tasks are added, so the slice can overshoot it by up to the size of one task. Deterministic in `seed`. ### tools\_from\_traces ```python theme={null} tools_from_traces(traces: Sequence[dict]) -> list[dict] ``` The agent's tool surface, read off the calls the traces contain. Argument names are unioned across every observed call, so a tool called with different arguments in different traces ends up with all of them. ### trace\_report ```python theme={null} trace_report( traces, tools: list[dict] | None = None, policy: str = '', ) -> dict[str, Any] ``` What these traces contain and what they will aim generation at. Run before `simulate(traces=...)`. With `tools` (and optionally `policy`) the report also computes the actual grid emphasis: which axis values move forward in the coverage grid because of these traces. Reward stays optional; ungraded counts are reported, never required. `advisory_labels` counts rows carrying only a `qwen_reward`: those labels do steer trace mining, so they are disclosed, not hidden under "ungraded". `dropped` counts input rows that carried no usable signal and were discarded by normalization.