MetaOthello: A Controlled Study of Multiple World Models in Transformers

Aviral Chawla^1,* Galen Hall^2,* Juniper Lovato¹

¹Vermont Complex Systems Institute, University of Vermont · ²University of Michigan

*Equal contribution

ICML 2026 · Seoul, South Korea

Foundation models must learn many generative processes, yet mechanistic interpretability largely studies capabilities in isolation; it remains unclear how a single transformer organizes multiple, potentially conflicting “world models”. Building from previous work, we introduce MetaOthello, a controlled suite of Othello-like games with varying rules, and train small GPTs on mixed-game data. We show that transformers trained on multiple Othello variants learn shared world-state representations: linear probes trained on one game intervene on another’s board state nearly as well as matched probes. When games conflict, the model resolves the resulting ambiguity through a localized mechanism we identify and steer. For isomorphic games with shuffled tokens, representations are equivalent up to a single orthogonal rotation that generalizes across layers, showing the shared structure is abstract rather than tied to surface form. Together, these results show that transformers reconcile conflicting world models by sharing structure and localizing conflict.

MetaOthello overview: an ambiguous move sequence is consistent with two different Othello variants (Classic and NoMidFlip); a transformer trained on both processes the sequence through shared early layers, and linear probes recover the correct, variant-specific board state at a later layer.

MetaOthello is a suite of Othello-like games played on the same 8×8 board, but follow different rules. We train small GPTs on sequences from mixtures of these games. We then probe the model to read — and intervene on — the board state it believes it is tracking. We show how the model learns to abstract across disparate games and, importantly, how it resolves ambiguity.

We Find

Shared Representations

Transformers learn to generalize shared abstractions. Board represenations for two disparate games is causally interchangeable.

Shared Computation

Models perform game-general computations in early layers and then later identify game identity for game specific calculations.

Ambiguity Circuit

When game sequences overlap and board representations are causally shared, we show mechanisms of how model resolve ambiguity.

Cite Us

If you find MetaOthello useful, please cite our paper:

@inproceedings{chawla_hall_2026_metaothello,
  title     = {MetaOthello: A Controlled Study of Multiple World Models in Transformers},
  author    = {Chawla, Aviral and Hall, Galen and Lovato, Juniper},
  booktitle = {Proceedings of the 43rd International Conference on Machine Learning},
  series    = {Proceedings of Machine Learning Research},
  volume    = {306},
  year      = {2026},
  address   = {Seoul, South Korea},
  publisher = {PMLR}
}