Large language models are running into limits in domains that require an understanding of the physical world — from robotics to autonomous driving to manufacturing. That constraint is pushing investors toward world models, with AMI Labs raising a $1.03 billion seed round shortly after World Labs secured $1 billion.

Large language models (LLMs) excel at processing abstract knowledge through next-token prediction, but they fundamentally lack grounding in physical causality. They cannot reliably predict the physical consequences of real-world actions. 

AI researchers and thought leaders are increasingly vocal about these limitations as the industry tries to push AI out of web browsers and into physical spaces. In an interview with podcaster Dwarkesh Patel, Turing Award recipient Richard Sutton warned that LLMs just mimic what people say instead of modeling the world, which limits their capacity to learn from experience and adjust themselves to changes in the world.

This is why models based on LLMs, including vision-language models (VLMs), can show brittle behavior and break with very small changes to their inputs. 

Google DeepMind CEO Demis Hassabis echoed this sentiment in another interview, pointing out that today’s AI models suffer from “jagged intelligence.” They can solve complex math olympiads but fail at basic physics because they are missing critical capabilities regarding real-world dynamics. 

To solve this problem, researchers are shifting focus to building world models that act as internal simulators, allowing AI systems to safely test hypotheses before taking physical action. However, “world models” is an umbrella term that encompasses several distinct architectural approaches. 

That has produced three distinct architectural approaches, each with different tradeoffs.

JEPA: built for real-time

The first main approach focuses on learning latent representations instead of trying to predict the dynamics of the world at the pixel level. Endorsed by AMI Labs, this method is heavily based on the Joint Embedding Predictive Architecture (JEPA). 

JEPA models try to mimic how humans understand the world. When we observe the world, we do not memorize every single pixel or irrelevant detail in a scene. For example, if you watch a car driving down a street, you track its trajectory and speed; you do not calculate the exact reflection of light on every single leaf of the trees in the background. 

JEPA models reproduce this human cognitive shortcut. Instead of forcing the neural network to predict exactly what the next frame of a video will look like, the model learns a smaller set of abstract, or “latent,” features. It discards the irrelevant details and focuses entirely on the core rules of how elements in the scene interact. This makes the model robust against background noise and small changes that break other models.

This architecture is highly compute and memory efficient. By ignoring irrelevant details, it requires much fewer training examples and runs with significantly lower latency. These characteristics make it suitable for applications where efficiency and real-time inference are non-negotiable, such as robotics, self-driving cars, and high-stakes enterprise workflows. 

For example, AMI is partnering with healthcare company Nabla to use this architecture to simulate operational complexity and reduce cognitive load in fast-paced healthcare settings. 

Yann LeCun, a pioneer of the JEPA architecture and co-founder of AMI, explained that world models based on JEPA are designed to be “controllable in the sense that you can give them goals, and by construction, the only thing they can do is accomplish those goals” in an interview with Newsweek.

Gaussian splats: built for space

A second approach leans on generative models to build complete spatial environments from scratch. Adopted by companies like World Labs, this method takes an initial prompt (it could be an image or a textual description) and uses a generative model to create a 3D Gaussian splat. A Gaussian splat is a technique for representing 3D scenes using millions of tiny, mathematical particles that define geometry and lighting. Unlike flat video generation, these 3D representations can be imported directly into standard physics and 3D engines, such as Unreal Engine, where users and other AI agents can freely navigate and interact with them from any angle.

The primary benefit here is a drastic reduction in the time and one-time generation cost required to create complex interactive 3D environments. It addresses the exact problem outlined by World Labs founder Fei-Fei Li, who noted that LLMs are ultimately like “wordsmiths in the dark,” possessing flowery language but lacking spatial intelligence and physical experience. World Labs’ Marble model gives AI that missing spatial awareness. 

While this approach is not designed for split-second, real-time execution, it has massive potential for spatial computing, interactive entertainment, industrial design, and building static training environments for robotics. The enterprise value is evident in Autodesk’s heavy backing of World Labs to integrate these models into their industrial design applications.

End-to-end generation: built for scale

The third approach uses an end-to-end generative model to process prompts and user actions, continuously generating the scene, physical dynamics, and reactions on the fly. Rather than exporting a static 3D file to an external physics engine, the model itself acts as the engine. It ingests an initial prompt alongside a continuous stream of user actions, and it generates the subsequent frames of the environment in real-time, calculating physics, lighting, and object reactions natively. 

DeepMind’s Genie 3 and Nvidia’s Cosmos fall into this category. These models provide a highly simple interface for generating infinite interactive experiences and massive volumes of synthetic data. DeepMind demonstrated this natively with Genie 3, showcasing how the model maintains strict object permanence and consistent physics at 24 frames per second without relying on a separate memory module.

This approach translates directly into heavy-duty synthetic data factories. Nvidia Cosmos uses this architecture to scale synthetic data and physical AI reasoning, allowing autonomous vehicle and robotics developers to synthesize rare, dangerous edge-case conditions without the cost or risk of physical testing. Waymo (a fellow Alphabet subsidiary) built its world model on top of Genie 3, adapting it for training its self-driving cars.

The downside to this end-to-end generative method is the great compute cost required to continuously render physics and pixels simultaneously. Still, the investment is necessary to achieve the vision laid out by Hassabis, who argues that a deep, internal understanding of physical causality is required because current AI is missing critical capabilities to operate safely in the real world.

What comes next: hybrid architectures

LLMs will continue to serve as the reasoning and communication interface, but world models are positioning themselves as foundational infrastructure for physical and spatial data pipelines. As the underlying models mature, we are seeing the emergence of hybrid architectures that draw on the strengths of each approach. 

For example, cybersecurity startup DeepTempo recently developed LogLM, a model that integrates elements from LLMs and JEPA to detect anomalies and cyber threats from security and network logs. 

Enterprises that have been juggling separate models for reasoning, multimodal tasks, and agentic coding may be able to simplify their stack: Mistral’s new Small 4 brings all three into a single open-source model, with adjustable reasoning levels under the hood.

Small 4 enters a crowded field of small models — including Qwen and Claude Haiku — that are competing on inference cost and benchmark performance. Mistral’s pitch: shorter outputs that translate to lower latency and cheaper tokens.

Mistral Small 4 updates Mistral Small 3.2, which came out in June 2025, and is available under an Apache 2.0 license. “With Small 4, users no longer need to choose between a fast instruct model, a powerful reasoning engine, or a multimodal assistant: one model now delivers all three, with configurable reasoning effort and best-in-class efficiency,” Mistral said in a blog post.

The company said that despite its smaller size — Mistral Small 4 has 119 billion total parameters with only 6 billion active parameters per token — the model combines the capabilities of all Mistral’s models. It has the reasoning capabilities of Magistral, the multimodal understanding of Pixtral, and the agentic coding performance of Devstral. It also has a 256K context window that the company said works well for long-form conversations and analysis.

Rob May, co-founder and CEO of the small language model marketplace Neurometric, told VentureBeat that Mistral Small 4 stands out for its architectural flexibility. However, it joins a rising number of smaller models that he said risks adding more fragmentation to the market. 

“From a technical perspective, yes, it can be competitive against other models,” May said. “The bigger issue is that it has to overcome market confusion. Mistral has to win the mindshare to get a shot at being part of that test set first.  Only then can they show the technical capabilities of the model.”

Reasoning on demand

Small models still offer good options for enterprise builders looking to have the same LLM experience at a lower cost.

The model is built on a mixture-of-experts architecture, much like other Mistral models. It features 128 experts with four active each token, which Mistral says enables efficient scaling and specialization.

This allows Mistral Small 4 to respond faster, even to more reasoning-intensive outputs. It can also process and reason about text and images, allowing users to parse documents and graphs. 

Mistral said the model features a new parameter it calls reasoning_effort, which would allow users to “dynamically adjust the model’s behavior.” Enterprises would be able to configure Small 4 to deliver fast, lightweight responses in the same style as Mistral Small 3.2, or make it wordier in the vein of Magistral, providing step-by-step reasoning for complex tasks, according to Mistral. 

Mistral said Small 4 runs on fewer chips than comparable models, with a recommended setup of four Nvidia HGX H100s or H200s, or two Nvidia DGX B200s.

“Delivering advanced open-source AI models requires broad optimization. Through close collaboration with Nvidia, inference has been optimized for both open source vLLM and SGLang, ensuring efficient, high-throughput serving across deployment scenarios,” Mistral said.

Benchmark performances

According to Mistral’s benchmarks, Small 4 performs close to the level of Mistral Medium 3.1 and Mistral Large 3, particularly in MMLU Pro.

Mistral said the instruction-following performance makes Small 4 suited for high-volume enterprise tasks such as document understanding.

While competitive with other small models from other companies, Small 4 still performs below other popular open-source models, especially in reasoning-intensive tasks. Qwen 3.5 122B and Qwen 3-next 80B outperform Small 4 on LiveCodeBench, as does Claude Haiku in instruct mode.

Mistral Small 4 was able to beat OpenAI’s GPT-OSS 120B in the LCR. 

Mistral argues that Small 4 achieves these scores with “significantly shorter outputs” that translate to lower inference costs and latency than the other models. In instruct mode specifically, Small 4 produces the shortest outputs of any model tested — 2.1K characters vs. 14.2K for Claude Haiku and 23.6K for GPT-OSS 120B. In reasoning mode, outputs are much longer (18.7K), which is expected for that use case.

May said that while model choice depends on an organization’s goals, latency is one of the three pillars they should prioritize. “It depends on your goals and what you are optimizing your architecture to accomplish. Enterprises should prioritize these three pillars: reliability and structured output, latency to intelligence ratio, fine-tunability and privacy,” May said.