The Brief
How AI Works 4 min read

Beyond Language: What World Models Are Actually Trying to Do

NAVION

Share

For the past few years, public understanding of artificial intelligence has been shaped almost entirely by large language models. Chatbots, text generators, coding assistants: these are the faces of AI that most people have encountered. A different category is now drawing serious attention, serious funding, and serious research effort. It goes by the name “world models,” and understanding what that term actually means, and what it does not mean, is more useful than following the announcements.

A Term That Means Several Things at Once

The first thing to understand about world models is that the label is contested. As Ben Mildenhall, co-founder of World Labs and co-creator of the neural radiance field (NeRF) technique, put it plainly: ask different companies what a world model is, and each will give a slightly different answer.

At its most general, a world model is any system that takes an interaction as input and simulates what would happen next in some environment. Runway, the video AI company, defines it as a system that builds an internal representation of an environment and uses that representation to simulate future events. Fei-Fei Li, the computer vision pioneer who co-founded World Labs, has proposed three criteria: the system generates outputs with perceptual, geometric, and physical consistency; it is multimodal by design; and it produces next states based on input actions.

Vincent Sitzmann, an assistant professor at MIT who leads the Scene Representation Group within CSAIL, offers a more grounded observation about where the field actually stands today. Most people using the term “world model,” he notes, are really talking about generating pixels: producing realistic video that responds to actions. That is a meaningful capability, but it is not the same as simulating physics or reasoning about space in a deep sense. The gap between the marketing language and the technical reality is worth keeping in mind.

How World Models Differ from LLMs, and Why That Matters

The distinction between world models and large language models is not just technical. It reflects a different theory of what intelligence requires.

LLMs operate on language. They are, as Fei-Fei Li wrote, “wordsmiths in the dark”: capable of producing eloquent, knowledgeable text, but without grounding in the physical world. Yann LeCun, former chief AI scientist at Meta, has argued that extending LLMs toward human-level intelligence is, in his words, “complete nonsense.” Clem Delangue, CEO of Hugging Face, a major platform for hosting language models, has suggested that the LLM bubble may be approaching its limits, while noting that AI applied to biology, chemistry, image, audio, and video is still in its early stages.

World models are designed to address this gap. Where an LLM interaction is turn-based, a text prompt followed by a text response, Mildenhall describes a world model as synchronous and real-time. The user or agent does not move through a linear sequence of exchanges. Instead, they interact with something that behaves like an environment, where multiple things happen in parallel and continuous actions are possible. This is a fundamentally different mode of engagement, closer to navigating a space than conducting a conversation.

The practical targets for world models reflect this difference. Researchers and companies in this space are focused on training and testing robots, generating three-dimensional assets for games and film production, and running scientific simulations. These are domains where language alone is insufficient and where the ability to model spatial relationships and physical dynamics is essential.

Why the Funding Surge Signals a Structural Shift

The commercial momentum behind world models is real. World Labs and AMI each reportedly raised around $1 billion in February and March respectively. Runway raised $315 million in February. These are not speculative bets on a distant future: they reflect conviction that specific, near-term applications are within reach.

This is what most coverage of world models misses. Unlike large language models, which began with a general-purpose interface (the chatbot) and then searched for use cases, the leading world model efforts are working in the opposite direction. They start with concrete problems: robot training, asset generation, scientific modeling. The interfaces and tools that will eventually deliver these capabilities to users are still being worked out.

The term “world model” itself has a longer history than the current wave of announcements suggests. It has appeared in reinforcement learning and robotics research for years, used to describe models that predict how environments change. What is new is the ambition to scale that idea into general-purpose generative systems trained on large volumes of visual and multimodal data.

In Short

World models are not a replacement for large language models. They are a different approach to a different problem: how to build AI systems that can reason about space, simulate physical dynamics, and interact with environments in real time rather than through text exchanges. The term is genuinely overloaded, used to describe everything from video generation to robot policy evaluation to 3D asset creation. The underlying conviction, shared by researchers at MIT, World Labs, and Runway, is that one foundational model architecture should eventually be able to address many of these problems. Whether that conviction proves correct is still an open question. What is already clear is that the field has moved from a research topic to a commercial priority, and the applications being built on top of it are grounded in practical needs, not abstract speculation.

Based on reporting from Ars Technica.

Written by

NAVION