> ## Documentation Index > Fetch the complete documentation index at: https://docs.withwhile.com/llms.txt > Use this file to discover all available pages before exploring further. # whileai.simulations.data > SimulationData: the object simulate() returns, and what it can do next. SimulationData: the object simulate() returns, and what it can do next. 6 public names. `import whileai.simulations as wai`, then `wai.name`. | Name | What it does | | ----------------------------------- | ----------------------------------------------------------------- | | [`SimulationData`](#simulationdata) | | | [`conversation`](#conversation) | User/agent turns from prompt + steps. | | [`grade`](#grade) | Binary 0/1 situation grade. | | [`grade_llm`](#grade_llm) | Binary 0/1 situation grade. | | [`llm_grade`](#llm_grade) | Module helper: advisory LLM scores on an existing SimulationData. | | [`rank`](#rank) | Score already-generated rows. | ### SimulationData ```python theme={null} SimulationData( trajectories: list[dict] = , arm_yield: dict = , stopped_because: str = 'budget', declared_tools: set = , stages: list[str] = , scaffold_chars: int = 0, degraded: list[str] = , warnings: list[str] = , semantic: bool = False, profile: AgentProfile | None = None, embedder_name: str = '', elapsed_seconds: float = 0.0, rows_per_second: float = 0.0, arm_weights: dict = , scenario_generation_seconds: float = 0.0, embedding_selection_seconds: float = 0.0, rollout_seconds: float = 0.0, row_seconds: list = , unique_prompts: int = 0, scene_brief: str = '', scene_brief_seconds: float = 0.0, first_row_seconds: float = 0.0, semantic_duplicate_rate: float | None = None, unique_behavior_signatures: int = 0, coverage_curve: list[dict] = , coverage: dict = , search: dict = , budget: int = 0, path: str = '', mode: str = 'explore', repeat_policy: str = 'none', n_situations: int | None = None, requests_per_situation: int = 1, rollouts_per_request: int = 1, unique_situations: bool = False, allocator: dict = , writer_model: str = '', user_model: str | None = None, system_prompts: dict[str, str] = , ) -> None ``` #### SimulationData.grade ```python theme={null} grade( self, grader=None, judge=None, llm: bool = False, llm_spec: str | None = None, api_key: str | None = None, path: str | None = None, concurrency: int = 32, llm_concurrency: int = 16, version: str | None = None, use_privileged: bool = False, scale: tuple[float, float] | None = None, rubric: str | None = None, trust: str = 'warn', payload_chars: int = 8000, max_tokens: int = 120, ) ``` Grade after simulation with the hosted judge or a custom callable. With no callable this is `grade_llm`: the hosted LLM judge (Phi-4, a different family from the hosted Qwen policy), read from `VLLM_API_KEY`. It writes `reward` and `reason` onto the rows in place and returns the judge report (a dict: graded, n0, n1, backend, judge\_version, warnings). `llm=True` is the same path. A plain `grader=` callable scores in place too and returns nothing. Simulation itself never invokes this method by default. `judge=` is the contract path: any callable honoring the judge contract (`judge(row) -> \{"reward": 0 or 1, "reason": str, "markers": \{name: value\}\}`; a bare number works too). The contract and its failure modes are written out in full in `whileai.simulations.score.judging` — note the `score.`, there is no `whileai.simulations.judging`. It returns a `ScoredData` of copies — trajectories here stay unmodified, judge errors are marked per-row instead of coerced to 0 — and its output feeds `export_training` and `simulate(traces=...)` directly. `version=` names the judge's version (model, rubric hash) and is recorded on every scored row; the hosted grader stamps its own. Every path then checks the judge against the rows' human labels (`attach_labels(kind="human")`) and stamps the summary on each graded row's `judge_meta["trust"]`. `trust="warn"` (default) logs one line when the check failed or no labels exist, `"require"` raises instead, `"off"` skips it. #### SimulationData.grade\_llm ```python theme={null} grade_llm( self, spec: str | None = None, base_url: str | None = None, model: str | None = None, concurrency: int = 16, api_key: str | None = None, path: str | None = None, limit: int | None = None, prompt: str | None = None, use_privileged: bool = False, rubric: str | None = None, trust: str = 'warn', payload_chars: int = 8000, max_tokens: int = 120, ) ``` Binary 0/1 situation grade. Default brain is the hosted judge (Phi-4 unless `WHILEAI_JUDGE` is set), never the policy model. `use_privileged` shows the judge each row's `privileged` block (principle, reference, hidden state) the agent never saw. `trust` is the judge check against human labels: see `grade`. `payload_chars` caps the evidence the judge reads per row and `max_tokens` its reply (defaults `JUDGE_PAYLOAD_CHARS` and `JUDGE_MAX_TOKENS` in `defaults.py`); both land in `judge_meta`. #### SimulationData.leak\_report ```python theme={null} leak_report(self, min_len: int = 12) -> dict[str, Any] ``` Did any reply quote its own `privileged` block? Reads the trajectories, which still carry the block; `rows()` is scrubbed and would check nothing. Same report as `leak_report`. #### SimulationData.llm\_grade ```python theme={null} llm_grade( self, spec: str | None = None, concurrency: int = 16, api_key: str | None = None, path: str | None = None, ) ``` Advisory LLM pass. Leaves deterministic reward untouched. #### SimulationData.push ```python theme={null} push( self, name: str, api_key: str | None = None, parent: str | None = None, agent: str | None = None, publish: bool = False, description: str | None = None, gate: bool = True, purpose: str = 'train', holdout: float | None = None, endorsed: Sequence[str] = (), strict_hacks: bool = False, ) -> dict ``` Upload this run to your While account as a dataset. `purpose` is the section it lands in on the Datasets page (`"train"` by default; `"holdout"` or `"eval"`). `holdout=0.2` keeps a fifth of the tasks (by `scenario_id`) out of the training set and pushes them as a second, linked dataset with purpose `"holdout"`; the entry carries it as `["holdout"]`. The simulation mode is recorded on both. `api_key` defaults to the `WHILEAI_API_KEY` env var, then the key saved by `whileai login`. Pass `parent` (a `ds_...` id) when this run iterates on an existing dataset, so lineage shows on the platform. `publish=True` with an `agent` name also puts it on the public catalog at zeroproofai.com/datasets as a card. Returns the registry entry with `datasetId`. `gate=True` runs `publish_gate` first: every graded row gets a `calibration` stamp (per-task pass rate, k, producing policy), and an RL-shaped run that is ungraded or has no mixed group is refused with `PublishGateError`. The gate report is returned as `entry["gate"]`. `gate=False` uploads rows as they are. `endorsed` names what the reward should track (feature-name substrings, e.g. `"tool:lookup_order"`) for the gate's `hack_scan`; `strict_hacks=True` refuses a set whose reward is best explained by something else. #### SimulationData.rank ```python theme={null} rank(self, path: str | None = None) -> dict ``` Second-pass quality scores. Leaves conduct `reward` untouched. Writes `quality`, `quality_reason`, `quality_scores` on each trajectory and rewrites the saved JSONL, or `path` if you pass one. #### SimulationData.report ```python theme={null} report(self) -> dict ``` Run-level coverage summary (same as `data.coverage`). #### SimulationData.select ```python theme={null} select(self, target: int = 1000) -> list[dict] ``` The rows recommended for training, not everything generated. Diverse pass-labeled demonstrations via `select_for_sft`: one of each distinct way of being right before any repeats, junk and duplicate prompts dropped. Requires graded rows — grade in-loop (`grade=True`, `grader=`) or afterwards with `grade()`. The selection report lands in `search["selection"]`. #### SimulationData.training\_set ```python theme={null} training_set( self, output: str | None = None, target: int = 1000, validate: bool = True, ) -> dict ``` Select the recommended rows and export them trainer-ready. `select()` picks diverse pass-labeled rows, `export_training` writes them as chat JSONL with this run's system prompt and tools and the tool-call round-trip gate. Returns the export report with the selection report attached; pass `output` to write the file. Raw simulation rows are not the training artifact — this is. ### conversation ```python theme={null} conversation(row: dict) -> list[dict] ``` User/agent turns from prompt + steps. Tool calls stay on the assistant turn. ### grade ```python theme={null} grade( source, spec: str | None = None, base_url: str | None = None, model: str | None = None, concurrency: int = 16, api_key: str | None = None, path: str | None = None, limit: int | None = None, output: str | None = None, prompt: str | None = None, policy: str = '', tools: list | None = None, use_privileged: bool = False, trust: str = 'warn', payload_chars: int = 8000, max_tokens: int = 120, ) ``` Binary 0/1 situation grade. Default brain is hosted Qwen. `source` is a `SimulationData`, a JSONL path, or a row list. Writes `reward` 0 or 1 and a one-sentence `reason`. Keeps the previous score as `qwen_reward` when present. Does not run during `simulate()`. Search does not read `reward`. `limit` grades that many rows then stops. Hosted Qwen reads `VLLM_API_KEY`. For a path or row list, pass `policy=` and `tools=` so the judge sees the agent's rules; a `SimulationData` supplies its own. `trust` is the judge check against human labels: see `SimulationData.grade`. ### grade\_llm ```python theme={null} grade_llm( source, spec: str | None = None, base_url: str | None = None, model: str | None = None, concurrency: int = 16, api_key: str | None = None, path: str | None = None, limit: int | None = None, output: str | None = None, prompt: str | None = None, policy: str = '', tools: list | None = None, use_privileged: bool = False, trust: str = 'warn', payload_chars: int = 8000, max_tokens: int = 120, ) ``` Binary 0/1 situation grade. Default brain is hosted Qwen. `source` is a `SimulationData`, a JSONL path, or a row list. Writes `reward` 0 or 1 and a one-sentence `reason`. Keeps the previous score as `qwen_reward` when present. Does not run during `simulate()`. Search does not read `reward`. `limit` grades that many rows then stops. Hosted Qwen reads `VLLM_API_KEY`. For a path or row list, pass `policy=` and `tools=` so the judge sees the agent's rules; a `SimulationData` supplies its own. `trust` is the judge check against human labels: see `SimulationData.grade`. ### llm\_grade ```python theme={null} llm_grade( data: SimulationData, spec: str | None = None, concurrency: int = 16, api_key: str | None = None, path: str | None = None, ) -> SimulationData ``` Module helper: advisory LLM scores on an existing SimulationData. ### rank ```python theme={null} rank( source, output: str | None = None, min_quality: float | None = None, ) -> dict ``` Score already-generated rows. `source` is a JSONL path, a row list, or a `SimulationData`. Does not change `simulate()` or `reward`.