Walking in the Implicit: Interactive World Exploration via Neural Scene Representation

Code will be released soon.

Zhiqi Li^1,2   Chengrui Dong^1,2   Zhenhua Du^1,2   Hangning Zhou^3,†   Cong Qiu³
Hailong Qin³   Mu Yang³   Dongxu Wei²   Peidong Liu^2,*

¹Zhejiang University    ²Westlake University    ³Afari Intelligent Drive
^†Project Lead    ^*Corresponding Author

News

• The public repository is under internal review. Code and checkpoints will be released soon.
• 🎉🎉 NeuWorld is accepted by ECCV 2026.

Highlights

• Scene-centric rollout. We replace growing video-latent trajectories with a fixed-length, renderable Neural Implicit Scene (NIS) state.
• Factorized interaction. Each step decouples stochastic latent scene-state transition from deterministic pose-conditioned rendering.
• Unified NIS conditioning. Camera, reference-image, and retrieved history cues are mapped into the same NIS modality instead of separate heterogeneous encoders.
• Long-horizon consistency. NeuWorld is designed for camera-controlled exploration with revisitation consistency and favorable inference efficiency.
• From-scratch training. The model is trained on public posed-view datasets without pretrained video backbones or auxiliary 3D reconstructors.

Method Overview

At each interaction step, the frozen NIS-VAE encoder maps the current observation and a sparse future pose trajectory to a partial NIS condition. Geometry-aware retrieval selects a history set and encodes it as memory NIS tokens. NIS-DiT samples the next local NIS state, and the frozen decoder renders future views under the queried poses.

Citation

If you find our work useful, please cite:

@inproceedings{li2026neuworld,
  title     = {Walking in the Implicit: Interactive World Exploration via Neural Scene Representation},
  author    = {Li, Zhiqi and Dong, Chengrui and Du, Zhenhua and Zhou, Hangning and Qiu, Cong and Qin, Hailong and Yang, Mu and Wei, Dongxu and Liu, Peidong},
  booktitle = {European Conference on Computer Vision (ECCV)},
  year      = {2026}
}