> ## Documentation Index
> Fetch the complete documentation index at: https://docs.withwhile.com/llms.txt
> Use this file to discover all available pages before exploring further.

# whileai.simulations.monitor

> Watch a live training run for reward hacking.

Watch a live training run for reward hacking.

2 public names. `import whileai.simulations as wai`, then `wai.name`.

| Name                                          | What it does                                          |
| --------------------------------------------- | ----------------------------------------------------- |
| [`HackMonitor`](#hackmonitor)                 | Watch a TRL run for reward hacking.                   |
| [`format_hack_monitor`](#format_hack_monitor) | The block a person reads: the curve, then the alarms. |

### HackMonitor

```python theme={null}
HackMonitor(
    run: TrainingRun | None = None,
    holdout: Sequence[Any],
    proxy: Callable[..., Sequence[float]] | None = None,
    gold: Callable[[dict], Any] | None = None,
    sample: Callable[..., list[list[str]]] | None = None,
    every: int = 10,
    k: int = 4,
    n_prompts: int = 32,
    window: int = 3,
    delta: float = 0.1,
    length_pct: float = 0.25,
    kl_budget: float | None = None,
    endorsed: Sequence[str] = (),
    stop_on: str | Sequence[str] = (),
    buffer: int = 512,
    max_new_tokens: int = 256,
    concurrency: int = 8,
    seed: int = 0,
    n_boot: int = 500,
    n_perm: int = 50,
    scan_min: int = 8,
    sampling: Mapping[str, Any] | None = None,
)
```

Watch a TRL run for reward hacking. See the module docstring.

`holdout` is a list of prompts (strings or message lists) or rows
(`{"prompt": ..., <extra columns>}`); extra columns reach the proxy
as keyword lists, the way TRL passes dataset columns. `proxy` is a
TRL-shaped reward function (`f(prompts=, completions=, **cols)`);
leave it `None` to use the reward function passed through
`wrap`. `gold` is a judge under the SDK contract (a rollout row
in, a reward or `{"reward": ...}` out). `sample` overrides how
completions are drawn: `sample(model, tokenizer, prompts, n=,
max_new_tokens=) -> list[list[str]]`; the default uses the chat
template and `model.generate`; `sampling` (`temperature`,
`top_p`, `batch`) steers that default sampler.

`n_boot` is the bootstrap count behind the gold-vs-window interval,
`n_perm` the permutation count behind the feature scan, `scan_min`
the fewest buffered completions the scan runs on. Every number has its
reason in `defaults.py` (MONITOR\_\*).

`run` is the platform run the points and alarms land on; `None`
keeps everything on the monitor (`history`, `alarms`,
`summary()`).

#### HackMonitor.evaluate

```python theme={null}
evaluate(
    self,
    step: int,
    model: Any = None,
    tokenizer: Any = None,
) -> dict[str, Any]
```

Sample the holdout from the live policy and score it both ways.
Appends to `history`, logs to the run, checks the alarms.
Returns the history entry.

#### HackMonitor.summary

```python theme={null}
summary(self) -> dict[str, Any]
```

What the run page and `finish` carry: the curve points, the
alarms, the last scan, where it stopped.

#### HackMonitor.wrap

```python theme={null}
wrap(
    self,
    reward_fn: Callable[..., Sequence[float]],
) -> Callable[..., Sequence[float]]
```

The reward function, watched: every completion it scores goes
into the buffer with its reward. Pass the result to the trainer
as `reward_funcs`. The name survives, so TRL's `rewards/<name>`
column does too.

### format\_hack\_monitor

```python theme={null}
format_hack_monitor(summary: Mapping[str, Any]) -> str
```

The block a person reads: the curve, then the alarms.