The Frontier of Physical AI: A Critical Appraisal of World Models

The pursuit of Artificial General Intelligence (AGI) is increasingly extending beyond language models operating in digital environments toward embodied systems capable of interacting with the physical world. Unlike software agents that reason solely through text or symbolic representations, autonomous robots must perceive, predict, and act within environments governed by geometry, dynamics, uncertainty, and the immutable laws of physics. This shift has placed world models at the center of contemporary physical AI research.

A capable autonomous agent cannot rely solely on reactive control or predefined heuristics. Before executing an action, it must anticipate how the surrounding environment is likely to respond. World models provide this predictive capability by enabling an agent to internally simulate future states — asking “What will happen if I perform this action?” before translating high-level intentions into physical movements.

As research has progressed, however, the once broad consensus surrounding world models has begun to fragment. What initially appeared to be a straightforward scaling problem has evolved into a collection of competing research directions, each built upon different assumptions about representation learning, data generation, and physical reasoning. At the same time, a deeper question has emerged: how should physical laws be represented within neural architectures? Increasing evidence suggests that advances in physical AI will depend not only on larger models or larger datasets, but also on how effectively these systems incorporate the structural constraints imposed by the physical world.

The central argument of this review is that the future of physical AI will be determined less by scaling neural architectures than by how effectively statistical learning is integrated with explicit representations of physical structure.

1. The Four Paths of Modern World Models

Research on world models has diversified into several distinct technical directions, each reflecting different assumptions about how an intelligent agent should represent the physical world, acquire knowledge, and support downstream decision making. Although these approaches share the common objective of enabling predictive reasoning, they differ substantially in their treatment of visual representation, abstraction, simulation, and scalability.

Figure 1: The Divergent Pathways of Physical AI. Genie, JEPA, GAIA, and Cosmos are four distinct technological roads branching from a core problem of physical AI.

The Genie Road: Learning Through Interactive Pixel Generation

Represented by Google DeepMind’s Genie family of models [1], this approach seeks to learn a world model directly from the richness of visual observations. Video sequences are tokenized and modeled using autoregressive or diffusion-based architectures, allowing the system to generate future visual states conditioned on an agent’s actions.

The principal strength of this strategy lies in its ability to produce visually realistic and interactive environments without requiring explicit geometric or physical supervision. However, visual realism does not necessarily imply physical correctness. Because the model learns statistical regularities rather than explicit physical principles, generated scenes may exhibit implausible behaviors, including violations of object permanence, unrealistic collisions, or failures to respect gravity and conservation laws.

The JEPA Road: Predictive Learning in Latent Space

Meta FAIR’s V-JEPA represents a markedly different philosophy [2]. Rather than reconstructing every pixel of an image, JEPA argues that many visual details contribute little to reasoning or planning. Features such as subtle lighting variations, reflections, or background texture consume model capacity without substantially improving an agent’s understanding of future events.

Instead, JEPA predicts future observations within a compact semantic latent space. By emphasizing abstract representations over pixel-level reconstruction, the model achieves greater sample efficiency and exhibits strong zero-shot generalization across unseen scenarios. The trade-off is reduced sensitivity to fine visual details, which can become important in applications requiring high-precision perception, such as dexterous manipulation or sub-millimeter robotic control.

The GAIA Road: Domain-Specific Synthetic Intelligence

Wayve’s GAIA framework [3] adopts a more specialized strategy by focusing exclusively on autonomous driving. Rather than pursuing a universally applicable world model, GAIA generates large volumes of high-quality synthetic data tailored to the complexities of road environments, including rare and safety-critical edge cases that are difficult to capture in real-world datasets.

This specialization enables highly optimized performance within its target domain. At the same time, it limits the broader applicability of the learned representations. Knowledge acquired for structured driving environments transfers poorly to other robotic settings, such as humanoid locomotion or industrial manipulation, where the underlying geometry, dynamics, and task requirements differ substantially.

The Cosmos Road: An End-to-End Industrial Pipeline

NVIDIA’s Cosmos ecosystem [4] extends the concept of a world model beyond a standalone predictive network into a complete simulation-to-deployment framework. By integrating synthetic data generation, World Action Models (WAMs), Isaac Lab, Omniverse, and the GR00T robotics platform, Cosmos provides an industrial pipeline for training and deploying physical AI systems at scale.

Its principal advantage is scalability. Synthetic environments can generate vast quantities of training data that would otherwise require expensive robotic experimentation. However, this scalability comes with increased system complexity. The framework depends heavily on proprietary software and hardware components, while its multi-stage architecture introduces numerous interfaces where accumulated modeling errors and integration inconsistencies may affect downstream performance.

Although these four research trajectories differ in philosophy and implementation, they address complementary aspects of the same challenge. Genie emphasizes visual generation, JEPA focuses on predictive abstraction, GAIA optimizes domain-specific intelligence, and Cosmos prioritizes large-scale deployment. Together, they illustrate that no single paradigm has yet emerged as the definitive architecture for physical AI, and each exposes different trade-offs between representation fidelity, computational efficiency, transferability, and physical realism.

2. Resolving Conceptual Overload: A Functional Taxonomy of World Models

As the term world model became increasingly widespread across academia and industry, it also became increasingly ambiguous. The phrase has been used to describe systems ranging from video generators to physics simulators and robotic planners, often without distinguishing their fundamentally different roles. This conceptual ambiguity has made it difficult to compare architectures or evaluate scientific progress.

To address this problem, Prof. Fei-Fei Li and colleagues at World Labs proposed a functional taxonomy [5] that separates world models into three complementary components, each responsible for a distinct stage of the perception–prediction–action pipeline.

Figure 2: The Decoupled Layers of a Unified World Model. The Functional Taxonomy reveals a vertically integrated stack of renderer, simulator and planner, showing how a Unified World Model (UWM) coordinates these three layers to achieve closed-loop physical AI.

The Renderer

The Renderer is responsible for generating observations. Its objective is to produce visually coherent images or videos that preserve spatial and temporal consistency, enabling either human interpretation or downstream visual processing. While rendering quality is essential for realistic simulation, the Renderer does not explicitly model the underlying mechanics that produce the observed scene.

The Simulator

The Simulator forms the physical core of the system. Rather than operating directly on pixels, it models the latent state of the environment, including geometry, object motion, collision boundaries, mass, friction, and other physical variables. It predicts how these quantities evolve over time, providing the causal structure required for accurate long-horizon forecasting.

The Planner

The Planner closes the perception–action loop. It receives the simulated state, evaluates candidate behaviors, and generates the sequence of actions most likely to accomplish the desired objective. In robotics, these outputs may correspond to control commands, trajectories, or action tokens that drive physical execution.

Toward a Unified World Model

The long-term objective is not simply to improve each component independently, but to integrate them into a Unified World Model (UWM). In such a system, simulation, rendering, and planning operate as tightly coupled processes rather than separate modules.

Given a high-level task specification, the Simulator predicts future physical states, the Renderer generates the corresponding observations, and the Planner continuously selects actions based on these evolving predictions. This closed-loop architecture enables an embodied agent to reason about future consequences before acting, allowing perception, prediction, and control to evolve together as part of a single decision-making process.

Viewed through this functional taxonomy, many apparent disagreements in the literature become less contradictory. Systems that appear to compete often emphasize different components of the same overall architecture. Some prioritize realistic rendering, others focus on accurate physical simulation, while still others concentrate on action planning. The broader challenge for physical AI is therefore not choosing among these functions, but integrating them into a coherent and computationally efficient framework capable of operating reliably in real-world environments.

3. The Core Dilemma: Embedding Physical Laws into World Models

Despite remarkable advances in generative modeling and representation learning, the central challenge of physical AI remains unresolved: how can a neural world model faithfully obey the laws of physics?

This question lies at the heart of the long-standing sim-to-real gap. A world model may generate visually convincing predictions, yet still fail catastrophically when deployed on a physical robot. Statistical learning alone offers no guarantee that the predicted evolution of a system will respect fundamental physical principles such as conservation of mass, momentum, energy, or geometric constraints including collision boundaries and mutual impassibility of two objects. As prediction horizons increase, even small violations can accumulate, eventually producing behaviors that are physically impossible.

The difficulty is therefore not merely one of improving prediction accuracy, but of determining how physical knowledge should be incorporated into neural architectures [6]. Current research has largely converged on two contrasting philosophies, each representing a different interpretation of the relationship between data-driven learning and first-principles physics.

Soft Constraints: Physics as an Optimization Objective

The most widely adopted strategy, exemplified by Physics-Informed Neural Networks (PINNs), treats physical laws as differentiable constraints that augment the training objective. Rather than altering the network architecture itself, conservation equations, governing differential equations, and boundary conditions are incorporated into the loss function,

This formulation is attractive because it is simple to implement and can be integrated into virtually any neural architecture. Existing machine learning pipelines require only modest modification, allowing physical knowledge to be introduced without redesigning the model.

Its flexibility, however, also exposes its fundamental limitation. Physical laws are encouraged rather than enforced. During optimization, the network seeks a compromise between fitting observational data and satisfying the imposed physical constraints. Compliance with physics therefore depends entirely on the optimization process rather than on any mathematical guarantee.

This limitation becomes particularly apparent when the objectives compete. The gradients associated with data fitting,

and those arising from the physical constraints

often point in different directions within the parameter space. The resulting gradient conflict slows convergence, complicates optimization, and may prevent the network from simultaneously satisfying both objectives.

The balance between these competing losses is controlled by the weighting coefficient (\lambda). Selecting an appropriate value is notoriously difficult. If the physical term is underweighted, the model achieves good predictive accuracy while violating conservation laws. If it is overemphasized, optimization becomes dominated by the physical constraints, limiting the model’s ability to capture the underlying data distribution.

These compromises become increasingly problematic during long-horizon prediction. Small inconsistencies introduced at each time step accumulate recursively, causing gradual drift away from physically admissible trajectories. A model that appears accurate over short intervals may therefore become unstable when required to reason over extended sequences or execute long-duration control policies.

Hard Constraints: Physics by Construction

An alternative philosophy seeks to eliminate these inconsistencies entirely by embedding physical principles directly into the model architecture. Rather than encouraging physically plausible behavior during training, hard-constrained models are designed so that violating specified physical laws is mathematically impossible.

This is achieved by restricting the network’s output space to a physically admissible manifold through carefully designed parameterizations, activation functions, or architectural structures. Because every prediction is generated within this constrained space, conservation laws and boundary conditions are satisfied exactly rather than approximately.

The resulting models typically exhibit substantially greater sample efficiency, since they do not expend learning capacity rediscovering constraints that are already known from physics. Exact enforcement also improves numerical stability and reduces the accumulation of physically inconsistent predictions during iterative simulation.

These advantages, however, come at a considerable cost.

Constructing physically constrained architectures generally requires problem-specific mathematical derivations that depend on the governing equations of a particular domain. A formulation developed for fluid dynamics rarely transfers directly to articulated robotics, deformable materials, or autonomous vehicles. As the diversity of robotic platforms increases, maintaining specialized architectures for each physical system becomes increasingly impractical.

Moreover, architectural constraints inevitably reduce modeling flexibility. Real-world environments rarely conform perfectly to idealized mathematical assumptions. Sensor noise, uncertain contact dynamics, unmodeled friction, manufacturing tolerances, and calibration errors introduce discrepancies that cannot always be represented within rigid constraint manifolds. Consequently, models designed to satisfy ideal physics exactly may become surprisingly fragile when confronted with imperfect observations collected outside controlled simulation environments.

Computational cost presents an additional challenge. Restricting optimization to constrained manifolds often increases memory consumption and computational overhead, limiting scalability for large Transformer-based world models and complex robotic systems.

The Fundamental Trade-off

Although soft and hard constraints are frequently presented as competing methodologies, they ultimately represent two ends of the same design spectrum.

Soft constraints maximize flexibility by allowing the model to learn from data while encouraging consistency with physical principles. Hard constraints maximize physical fidelity by embedding those principles directly into the model’s mathematical structure. Neither approach provides a complete solution. One sacrifices certainty for adaptability; the other sacrifices adaptability for certainty.

This tension has become one of the defining challenges in modern physical AI. Increasing evidence suggests that future world models will not rely exclusively on either philosophy. Instead, the field is moving toward architectures that combine the strengths of both approaches while mitigating their respective weaknesses — a direction explored in the next section.

4. Beyond the Soft–Hard Dichotomy: Emerging Architectures for Physics-Aware World Models

The limitations of both soft and hard constraints have prompted researchers to reconsider a more fundamental question: must physical reasoning be encoded through constraints at all? Increasingly, the answer appears to be no. Rather than refining existing paradigms, recent work has begun to redesign the architecture of world models themselves, introducing structural innovations that embed physical knowledge more naturally while avoiding many of the optimization and scalability issues associated with conventional approaches.

Two representative directions have emerged. One integrates physical structure directly into the network’s internal representation, while the other separates visual generation from physical simulation altogether. Although these strategies differ substantially in implementation, both seek to move beyond the traditional compromise between optimization flexibility and architectural rigidity.

Ψ-NN: Learning Within Physically Structured Representations

The Ψ-NN (Physics Structure-informed Neural Network) framework rethinks the role of physical knowledge in neural learning [6]. Instead of introducing physical laws as additional optimization objectives, Ψ-NN incorporates them directly into the model’s internal parameterization through a process of distillation, extraction, and remodeling.

In this framework, physical symmetries are treated as intrinsic structural properties rather than external constraints. For example, reflectional symmetry, u(x,y)=u(x,-y), is embedded into the network’s weight matrices instead of being enforced through a penalty term in the loss function. Consequently, the network is constructed so that these symmetries are preserved automatically throughout training.

This architectural design fundamentally changes the optimization landscape. Because the desired physical invariances are already encoded in the model structure, the competing objectives responsible for gradient conflict in conventional PINNs largely disappear. Standard backpropagation can therefore optimize the remaining learnable parameters without continually negotiating between data fidelity and physical consistency.

Experimental studies have demonstrated substantial improvements in predictive accuracy, with reductions in full-field error approaching 95% for representative benchmark problems. More importantly, Ψ-NN illustrates a broader conceptual shift: physical laws need not remain external supervisory signals — they can instead become part of the network’s internal representation, shaping the hypothesis space before learning even begins.

Dual Representations: Separating Vision from Physics

A second line of research questions a different assumption. Rather than asking how neural networks should learn increasingly accurate approximations of physical dynamics, it asks whether they should approximate these dynamics at all.

This philosophy is exemplified by World Labs’ Marble architecture [7], which adopts a dual-representation framework that cleanly separates visual realism from physical reasoning. Instead of requiring a single neural representation to satisfy both objectives simultaneously, Marble generates two complementary outputs, each optimized for a different purpose.

Figure 3: Dual-Representation. A visual image illustrates an elegant utility of dual-representation, showing how a model can render beautiful pixels using neural models while computing exact physics using traditional solvers.

The first is a high-fidelity rendering representation based on Gaussian Splats, which captures appearance, geometry, lighting, and photorealistic scene reconstruction. This representation is optimized for visual quality and perceptual realism.

The second is an explicit geometric representation composed of GLB collision meshes. Unlike neural latent features, these meshes describe the physical structure of the environment in a form directly compatible with conventional simulation engines.

This separation allows physical interactions — including contact, collision detection, friction, and rigid-body dynamics — to be computed by mature deterministic physics engines such as MuJoCo or PhysX rather than approximated by a neural network. The neural model is therefore responsible for generating realistic environments, while established physics solvers handle the mechanics of interaction.

The significance of this design extends beyond improved simulation accuracy. It recognizes that neural networks and classical physics engines possess fundamentally different strengths. Neural models excel at learning complex visual representations from large-scale data, whereas deterministic simulators remain considerably more reliable for enforcing geometric consistency and mechanical laws. By assigning each component the task for which it is best suited, the overall system achieves greater robustness than either approach alone.

A Shift in Design Philosophy

Although Ψ-NN and Marble are architecturally very different, they share an important conceptual insight. Both reject the traditional view that physical knowledge should simply be appended to a neural network as an additional optimization objective.

Ψ-NN internalizes physical structure within the network itself, whereas Marble externalizes physical computation to deterministic simulation engines. In both cases, physics becomes an integral part of the computational architecture rather than a secondary regularization term.

This emerging perspective represents a significant departure from earlier generations of physics-informed learning. Instead of asking how strongly physical constraints should be imposed during optimization, researchers are increasingly asking where physical reasoning should reside within the overall system architecture. The answer to that question is likely to shape the next generation of world models for embodied intelligence.

5. Future Directions for Physics-Aware World Models

As world models evolve from research prototypes toward practical deployment in embodied intelligence, the integration of physical reasoning is becoming a central design consideration rather than an auxiliary feature. Current developments suggest that progress will depend less on scaling individual models and more on combining complementary sources of physical knowledge within unified computational frameworks.

Several research directions are likely to define the next stage of development.

Hybrid Constraint Architectures

Rather than viewing soft and hard constraints as mutually exclusive alternatives, many researchers now regard them as complementary mechanisms operating at different levels of abstraction. Fundamental conservation laws — including mass, momentum, and energy — are well suited to exact architectural enforcement, while complex phenomena that are difficult to model analytically, such as turbulent flow, compliant materials, or uncertain contact dynamics, may be better handled through adaptive optimization objectives.

Hybrid architectures seek to combine these strengths, providing rigorous physical guarantees where they are essential while preserving sufficient flexibility to accommodate the variability of real-world environments.

Automated Discovery of Physical Structure

Designing physically constrained neural architectures currently requires substantial domain expertise and problem-specific mathematical derivations. Future systems are expected to automate much of this process by identifying the governing structural relationships directly from data, robot morphology, and task specifications.

Such approaches could significantly reduce the engineering effort required to construct physics-aware models while improving their adaptability across diverse robotic platforms.

Adaptive Constraint Enforcement

Real-world environments rarely satisfy idealized mathematical assumptions. Consequently, future world models are likely to adjust the strength of physical constraints dynamically rather than treating them as fixed throughout inference.

By estimating uncertainty from sensory observations and predictive confidence, these models could relax or reinforce individual constraints according to the reliability of the available information. Such adaptive mechanisms may provide a practical compromise between strict physical consistency and robustness under imperfect sensing conditions.

Closed-Loop Evaluation and Unified Benchmarks

Traditional benchmarks have primarily evaluated world models through open-loop prediction accuracy. While useful, these metrics often fail to reveal error accumulation during sequential decision making.

Future evaluation protocols will increasingly emphasize closed-loop interaction, measuring how prediction quality influences downstream planning, control, and task execution over extended time horizons. Comprehensive benchmarks that jointly assess physical consistency, visual realism, and policy performance are likely to replace isolated performance metrics, providing a more realistic assessment of embodied intelligence.

Integrating Structural Physics with Foundation Models

An important open challenge is extending recent structural innovations to large Transformer-based world models. Unlike conventional neural architectures, Transformer attention mechanisms do not readily accommodate explicit mathematical constraints or symmetry-preserving parameterizations.

At the same time, differentiable physics engines are becoming increasingly compatible with latent-space prediction models, creating opportunities to integrate first-principles simulation directly into foundation models for physical AI. Successfully combining large-scale representation learning with rigorous physical reasoning remains one of the most promising — and technically demanding — research directions in the field.

Looking Ahead

Taken together, these developments point toward a broader transformation in how world models are conceived. Future systems are unlikely to rely exclusively on larger datasets or increasingly powerful neural architectures. Instead, progress will depend on integrating statistical learning, explicit physical knowledge, and deterministic simulation into coherent computational frameworks capable of reasoning reliably about the physical world.

Rather than replacing classical physics with deep learning, the next generation of world models is increasingly being designed to leverage the strengths of both.

Conclusion

The evolution of world models reflects a broader transformation in the pursuit of embodied intelligence. Early research largely emphasized richer visual generation and larger predictive models, driven by the expectation that scaling data and model capacity would naturally yield increasingly capable physical reasoning. Recent advances suggest a more nuanced reality. Accurate prediction alone is insufficient when intelligent agents must operate within environments governed by immutable physical laws.

This realization has shifted the central challenge of physical AI from representation to integration. The question is no longer simply how to build more expressive neural networks, but how to combine statistical learning with the structural regularities of the physical world. The diverse research directions surveyed in this review — including generative world models, latent predictive architectures, functional taxonomies, physics-informed learning, and emerging dual-representation systems — can all be viewed as different attempts to resolve this fundamental problem.

The comparison between soft and hard constraint methodologies illustrates that neither purely data-driven optimization nor rigid analytical enforcement offers a complete solution. Flexible neural models adapt well to complex environments but provide no guarantee of physical consistency, whereas mathematically constrained architectures achieve exact compliance at the expense of scalability and generality. Increasingly, the field is moving beyond this dichotomy toward hybrid computational frameworks that combine learned representations, explicit physical structure, and deterministic simulation within a unified decision-making process.

Perhaps the most important insight emerging from recent research is that physical reasoning need not reside exclusively within a neural network. Instead, it can be distributed across multiple computational representations, with neural models, geometric structures, and classical physics engines each contributing according to their respective strengths. This architectural perspective marks a significant departure from viewing deep learning as a universal replacement for traditional modeling approaches. Rather than competing with established physical simulation, modern world models increasingly seek to complement and integrate it.

The long-term success of physical AI will therefore depend not only on advances in machine learning, but also on our ability to design systems whose computational structure reflects the organization of the physical world itself. Achieving reliable embodied intelligence requires more than predicting what is likely to happen — it requires ensuring that every prediction remains compatible with the fundamental principles that govern reality.

Ultimately, the future of world models is unlikely to be defined by ever-larger neural networks alone. It will be shaped by architectures that unite data-driven intelligence with physically grounded computation, enabling autonomous systems to reason, plan, and act with both the flexibility of learning and the reliability of physics.

References

[1] Google DeepMind. Genie 3: A new frontier for world models. 2025. https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/

[2] Meta FAIR. V-JEPA 2: Self-Supervised Video Models. 2025. https://ai.meta.com/research/vjepa/

[3] Wayve. GAIA-3: Scaling World Models. 2025. https://wayve.ai/thinking/gaia-3/

[4] NVIDIA. Cosmos World Foundation Model Platform. 2025. https://www.nvidia.com/en-us/ai/cosmos/

[5] Fei-Fei Li et al. A Functional Taxonomy of World Models. World Labs, 2026. https://drfeifei.substack.com/p/a-functional-taxonomy-of-world-models

[6] Chen-an Zhang, et al. Automatic network structure discovery of physics informed neural networks via knowledge distillation, Nat. Comm. 16, 9558 (2025).

[7] World Labs, How Marble Is Transforming Architectural and Interior Visualization?