When can a generative model be edited like a simulator through its latent state?

Editability asks whether a write changes the next prediction

A simulator holds an explicit state, advances it by a rule, and renders it into observations. A generative model trained on those observations has internal activations but no built-in way to set the world's state. The paper calls the missing property editability: whether changing the model's latent representation produces the next-step output you would get by editing the simulator's state.

The test is deliberately narrow. The model is trained on observations alone and frozen. A simulator state is edited to a target, an editor proposes a matching latent state, and the model's decoder turns that latent into a predicted next observation. That prediction is compared with the observation rendered from the edited simulator state advanced one step. Editability is therefore a claim about accessible simulator-like function, not about whether the model internally represents or advances state the way a conventional simulator does.

Two editors are compared. Representation steering trains a probe to decode simulator state from the latent state, then inverts or follows its gradients to move the latent. Inverse mapping instead learns a direct map from simulator state to latent state. Neither editor is trained on the observations its edits should produce, which keeps the comparison from being solved by supervision on the target output.

Direction and whole-output accuracy are scored separately

The paper separates two things an edit can do. Edit Index measures whether the prediction moves in the right direction, and it is computed only on the part of the output the edit should change: the rays whose values differ between the two worlds in Rayworld, and the cells whose legality the flip changes in Othello. It runs from −1, matching the unedited observation, to +1, matching the edited one.

Edit Fidelity measures whether the whole output stays accurate relative to the unedited prediction. Zero means no better than making no edit, negative values are worse, and 1 reproduces the edited observation exactly. An editor can implement the target change while degrading the rest of the output, or improve the output without implementing the change, so the two numbers answer different questions.

Decodability is measured separately with Probe Skill, a baseline-relative score where a probe's held-out loss is compared with a predictor that ignores the model. For continuous targets this reduces to out-of-sample R²; for categorical targets it uses error rate against the most common class. The paper also fits the same probes to a randomly initialized model and to raw observation sequences, so training's contribution can be separated from what a probe extracts regardless.

Othello editability falls smoothly as the rules are simplified

Othello is played on an 8×8 board, and the paper follows a setup in which every move is drawn uniformly from the legal set, so the optimal next-move prediction is a uniform distribution over legal moves. The simulator state is the board after each move, with each cell empty, mine, or theirs relative to the player about to move. An edit flips the color of one token after a real 20-move opening.

Every editor edits standard Othello, carrying the predicted move distribution most of the way to the flipped board. Every rule change breaks this. The paper introduces three variants: adjacent-flip allows placement anywhere adjacent to one's own color with flipping still active; adjacent-noflip uses the same placement rule with flipping off; standard-noflip keeps standard placement without flipping, so each game fills the same checkerboard in a different order.

With flipping off, no editor lands on either no-flip variant, even on adjacent-noflip, where legal moves still depend on token color. Adjacent-flip is where the editors separate: the inverse map still lands, while probe inversion and gradient steering reach at most +0.35 at low fidelity. The authors describe this as access to the model's state narrowing gradually rather than disappearing at once.

Decodability does not explain the pattern. The simulator state is decodable in every variant, with the MLP probe reaching a Probe Skill of at least 0.88. In Othello, training adds something the baselines lack only when flipping is on; once flipping is removed the board can be read off the move sequence itself, and every probe on standard-noflip scores a perfect 1.000, even on a randomly initialized model.

Rayworld probes fail where the inverse map still lands

Rayworld is the paper's own continuous world: two discs move at constant velocity for 40 frames inside a trapezoidal frustum, and each frame is rendered into a 1D observation by casting 128 rays. A ray returns the reflectivity of the nearest disc it hits, or 0 if it hits neither. Depth is never observed directly; it enters only through how many rays a disc covers and through occlusion. The simulator state holds the position and velocity of both discs.

On continuous positions, the probe-derived editors fail on every Rayworld variant while the inverse map lands everywhere, including three further variants reported in the appendix. Probe inversion's write reaches its target in probe space: at a step size of 1 the linear probe reads the edited state exactly, yet the predicted frame does not move. A latent state that renders the teleported disc therefore exists and can be written, but not along any direction the probes provide.

Discretizing the target changes this. It raises gradient steering on the 128-ray model, and as the rays coarsen both probe-derived editors rise steadily, to +0.51 and +0.60 at 5 rays. Because the number of bins falls with the ray count, the paper notes this trend cannot separate coarseness from target size. At 8 rays, evenly spaced bins that ignore the observation also improve on continuous positions, but bins defined in the appearance space improve most.

The authors interpret this as evidence that the model never forms a single abstract representation of where the discs are. The inverse map, by contrast, sets continuous positions at almost every residual point and still partially succeeds when the discs move as a rigid pair and an edit breaks their fixed spacing.

A weak latent fit can still support successful edits

In Rayworld the inverse map's fit and its edits separate. It explains far less of the latent variance, with R² between 0.32 and 0.58, yet it lands everywhere. Because its edits still land despite the weak fit, the paper argues that much of the latent variance appears to have little bearing on the prediction. The edited predictions are visibly smoother than the model's own, consistent with the inverse map producing a simplified latent that omits some of that variance.

The Othello results point in a different direction. There the inverse map lands only at residual points 4 and 5, which the authors read as the model funneling its history through a board-like bottleneck that the Rayworld model never forms. Whether a model must form such a bottleneck depends on what the environment demands of prediction, and the Othello variants vary exactly that demand.

This is the paper's central comparison: decodability is present in every variant, but which editor succeeds depends strongly on the environment. A model can be driven by a simulator it may have never learned, but the route to that control is not the same across discrete board states and continuous rendered scenes.

The claim covers one step in one architecture

Editability here is a statement about a single step. A simulator also carries its state forward under consistent rules, and the paper tests only whether one write changes the next prediction as it should. Rewriting the history so an edit persists through a rollout is demonstrated in the appendix, and extending the metrics to rollouts is named as the natural next step.

Every model is one transformer at one depth, so some findings may be a property of that architecture or depth rather than of the environments; recurrent or diffusion models may behave differently. The environments also differ in difficulty in ways the paper cannot fully control, and a harder environment may be less editable for reasons unrelated to how its state is represented.

The paper reports one training run per variant, with two further seeds giving a seed standard deviation of at most 0.004 for every trained probe's Probe Skill and below 0.04 for the Edit Index of every editor that lands. Cells whose spread exceeds 0.1 are marked, and all but the inverse map's fidelity on adjacent-flip are failing editors whose selected setting flips between seeds at the fidelity cutoff.

Choose the editor by environment, not by decodability

The practical priority follows from the split the paper documents. When the environment is a discrete board whose rules keep color relevant to prediction, probe-based steering is a reasonable first route, and standard Othello is edited by every editor tested. When the rules are simplified so that flipping is off, no editor sets the board, even though the state remains decodable.

When the state is continuous and revealed through partial renderings, the probe route should not be trusted on the strength of decodability alone: probe inversion can hit its target in probe space while the predicted frame does not move. The inverse map is the editor that lands on continuous positions, and its weak R² is not by itself a reason to reject it, because its edits still land.

The decision threshold the paper supports is therefore about the environment and the state description, not about how well a probe decodes. Discretizing the target is what revives probe-based steering in Rayworld, and the improvement grows as the rays coarsen, to +0.51 and +0.60 at 5 rays. The most valuable reading is that a model can be driven from a simulator's state even when its probes cannot steer it, and that this route should be tested per environment rather than assumed from decodability.

Sources used in this answer