EAI6 Talk: Geometry and Physics Bias in Embodied AI

Jiayun (Peter) Wang of Caltech will also be talking at Embodied AI 6 on the topic of Simulation for Embodied AI:

Geometry and Physics Bias in Embodied AI

Embodied AI demands agents that see the world with geometric fidelity, anticipate and interact with it with physical rigor. The talk will present a three-stage ladder — Perceive, Predict, Control — showing how carefully chosen geometry and physics biases enable that climb with minimal supervision. 1) Perceive. Pose-Aware Self-Supervised Learning learns semantic and geometric features from unlabeled videos. By regularizing along the agent’s own viewpoint trajectory, the network acquires a 3-D understanding without a single human label. 2) Predict and control. Controlling aerodynamic forces in turbulent conditions is crucial for UAV operation. We show AI enables realtime fluid flow prediction and turbulence control for wall friction reduction, which outperforms existing methods requiring expensive simulations of turbulent fluid dynamics. We further close the loop with FALCON, a model-based reinforcement learning framework for effective modeling and control of aerodynamic forces under turbulent flows. FALCON learns to control the underlying nonlinear dynamics when tested in the Caltech wind tunnel under highly turbulent conditions. Together, these works illustrate a unifying recipe: geometry grounds perception, physics grounds prediction and their composition unlocks fast, sample-efficient control.

Jiayun (Peter) Wang is a postdoctoral researcher at the California Institute of Technology, working with Prof. Anima Anandkumar. He received his PhD from UC Berkeley in 2023, advised by Prof. Stella Yu. His research develops novel machine learning and computer vision methodologies that address challenges of data scarcity and computational cost, with real-world applications like healthcare. More information can be found at his website: https://pwang.pw.