OccSim: Multi-kilometer simulation
with long-horizon occupancy world models

Tianran Liu, Shengwen Zhao, Mozhgan Pourkeshavarz, Weican Li, Nicholas Rhinehart

Learning, Embodied Autonomy, and Forecasting (LEAF) Lab, University of Toronto

Arxiv Code

World Model Driven Simulator

As far as we know, OccSim is the first occupancy world model driven simulation via autoregressive rollouts.

HD-Map free Generation

By eliminating the reliance on labeled HD maps during simulation, OccSim enables scalable AD simulation.

SoTA Generation Length & Quality

W-DiT enables at least 80x longer static semantic occupancy generation with SoTA fidelity and diversity.

Towards next Generation Sim.

Data collected from OccSim enable 22.1% improvement on 0-shot dynamic prior learning over asset-based Sim.

Over 3000 frames static semantic occupancy
rollouts with W-DiT for diverse road map construction

Tested on trajectory of varying difficulty (Straight, Zigzag, Closed-loop),
W-DiT structure presents the best stability and performance compare with previous design.

Stable Rollout Length (Higher is better)

3000
OccSim (Ours)
<40
Previous SoTA

Worst FID Score at 1000th frame under different trajectory (Lower is better)

62.5
OccSim (Ours)
183.6
Previous SoTA

Conditioned on the initial frame from various scenes in the Occ3D-nuScenes semantic occupancy dataset, we evaluate W-DiT's rollout capabilities across diverse driving trajectories (as illustrated in the animation on the right). Feel free to explore the three distinct scenarios visualized below. Notably, our model maintains robust generation stability even across long-horizon rollouts spanning thousands of frames.

Straight
Zigzag
Closed-loop
Start
GT (1-40)
Extended
Scene-039
Scene-221
Scene-632

Towards the next generation of Occupancy WM-centric simulation:
SoTA generation diversity & downstream pretrain

Given the same initial frame and the same straight trajectory, our model present SoTA generation diversity compared with previous occupancy world model. After 50s (100 frames, we use 6m/s speed to generate trajectory)
rollout, the branch road topology, vegetation / man-made presents obvious difference (random seed 999 / 2026 / 3407). Hover over the images to enlarge for better details.

Scene-0039

Initial Frame & Multi-seed Rollouts

Initial Frame Initial Frame
5s10s25s50s
Seed 999
Seed 2026
Seed 3407

Scene-0105

Initial Frame & Multi-seed Rollouts

Initial Frame Initial Frame
5s10s25s50s
Seed 999
Seed 2026
Seed 3047

HD-map-free generation enables the scaling of driving scenario and construct complex road networks. By integrating real-world agent assets and control algorithms onto this foundation, we facilitate effective pre-training for occupancy forecasting models. Models trained on data synthesized by OccSim (pre-trained on Waymo Occ) significantly outperform those trained on equivalent Carla datasets in 0-shot tests on UniScenes. Even in the worst-case scenario (OccWorld), data collected from OccSim provides an 11% performance boost over Carla data, which further scales to 22.1% with 5x data expansion.

OccWorld 3s average IoU (Deterministic Model)

18.08
OccSim (Ours 5x)
16.41
OccSim (Ours)
14.81
Pre-train on Carla

OccFM 3s average IoU (Stochastic Model)

21.40
OccSim (Ours 5x)
14.72
OccSim (Ours)
6.94
Pre-train on Carla

Acknowledgement & citation

This research was enabled in part by the following programs; we appreciate the resources provided:

  • NVIDIA Applied Research Program: We gratefully acknowledge the generous computes support, which significantly accelerated our model training.
  • Digital Research Alliance of Canada: For providing the computational infrastructure and Nibi, Trillium and Narval cluster.
  • Google TPU Research Cloud: For additional TPU resources grants at Google GCP.

Please use following format to cite our work if you find it's interesting

@article{liu2026occsim,
  title={OccSim: Multi-kilometer Simulation with Long-horizon Occupancy World Models},
  author={Liu, Tianran and Zhao, Shengwen and Pourkeshavarz, Mozhgan and Li, Weican and Rhinehart, Nicholas},
  journal={arXiv preprint arXiv:2603.28887},
  year={2026}
}