Physical AI
How learned intelligence moves into physical action, why state transitions make robot data difficult, and how the field shifted robot behavior out of engineered code and into datasets and learned parameters.
Embodied AI
How embodied agents learn through the closed loop of perception, action, and consequence, why data trustworthiness rather than data volume decides whether a policy transfers, and how pooled cross-embodiment corpora changed the field.
Summary
Embodied AI studies systems that learn by acting. An agent perceives its surroundings, takes an action, observes how the world changed, and uses that change as the input to its next decision. That loop is what separates it from models trained on static datasets, because nothing arrives pre-labeled and every mistake alters the situation the agent then has to solve. The central tension of the field is that describing an action correctly and executing it robustly are different capabilities, and the data that teaches the second is far scarcer than the data that teaches the first.
Embodied AI: Definition and Scope
Embodied AI covers agents whose intelligence is expressed through a body acting in an environment, physical or simulated. It includes navigation, manipulation, whole-body control, and the embodied reasoning that decides what to do before anything moves. The defining property is the closed loop: the agent’s own actions change the distribution of observations it will see next, so training data and evaluation both have to be sequential rather than independent samples.
Both physical and virtual embodiment count. A policy learned in a photorealistic simulator such as Habitat is embodied in the sense that matters academically, because the agent still acts and still receives consequences. What falls outside the field is passive perception, where a model labels an image and nothing follows from the label, and disembodied assistants that produce text with no action space. A teleoperated arm with no learned policy is also outside it, though it is one of the most important data sources feeding it.
The academic and industrial senses of the term differ and are worth separating. In research the phrase descends from embodied cognition and from Rodney Brooks, whose Intelligence without representation (Artificial Intelligence 47, 1991) argued that competent behavior can emerge from tight body and environment coupling rather than from rich internal world models. In industry the phrase increasingly means robot foundation models and the products built on them. This node uses the research sense and treats the industrial one as a subset. Its sibling hub Physical AI approaches the same territory from the deployment side, asking what it takes for learned intelligence to run reliably on real machines, and the two are best read as one field described from two ends.
How the Field Arrived Here
| Period | Milestone | Significance |
|---|---|---|
| 1991 | Brooks publishes Intelligence without representation | Establishes the argument that intelligence is grounded in interaction, not in symbolic world models |
| 2018 | Room-to-Room vision-and-language navigation, CVPR 2018 | Puts language-conditioned navigation into photorealistic scanned buildings, creating a reproducible embodied benchmark |
| 2019 | Habitat, ICCV 2019 | Simulation fast enough to train and compare embodied agents at scale, which made results comparable across labs |
| 2020 | ALFRED, CVPR 2020 | Joins navigation and manipulation under natural language, with 25K annotations across 120 indoor scenes |
| 2022 | SayCan | Grounds language model planning in what a specific robot can actually do, separating intent from affordance |
| 2022 | RT-1 | A transformer policy trained on large-scale real demonstrations rather than per-task engineering |
| 2023 | RT-2 | Vision-language pretraining transfers into action, defining the Vision-Language-Action model class |
| Oct 2023 | Open X-Embodiment | Pools 60 datasets from 34 labs into over 1 million trajectories across 22 embodiments, proving cross-embodiment transfer |
| 2024 | OpenVLA | An openly released VLA gives the field a reproducible baseline instead of proprietary checkpoints |
| Apr 2025 | π0.5 | Multi-stage manipulation lasting 10 to 15 minutes in homes never seen during training |
| Sep 2025 | Gemini Robotics 1.5 | Splits the stack into an embodied reasoning model that plans and a VLA that executes |
| Apr to Jul 2026 | Gemini Robotics-ER 1.6 (14 Apr 2026), π0.7 (16 Apr 2026), Gemini Robotics ER 2 (Jul 2026) | Embodied reasoning becomes an API-available component, and steerable policies begin generalizing zero-shot to unseen tasks |
The pattern the table reveals is a steady migration of the unit of progress. It moved from one lab’s controller, to one lab’s benchmark, to a pooled corpus and a shared model that many labs draw from. Every jump in capability since 2022 came from enlarging and standardizing the data, not from a new control theory.
Why Now
Pooled cross-embodiment data. Open X-Embodiment assembled over 1 million real trajectories spanning 22 embodiments and 527 skills from 34 institutions, and showed that a policy trained on the pool outperforms policies trained on any single robot’s data. That result changed what a robotics dataset is for, because contributing data now buys capability back.
Transfer from vision-language pretraining. Because a VLA inherits a web-scale visual and linguistic prior, it does not have to learn what a mug is from robot demonstrations. π0.5 uses this to reach homes outside its training distribution, and π0.7 reports zero-shot cross-embodiment behavior such as folding laundry it had not been shown. This factor is downstream of the previous one rather than parallel to it: the prior is only useful once there is enough action data to attach it to.
Simulation that is fast and realistic enough to train in. Habitat and successors made it possible to run millions of embodied episodes cheaply, which turned Sim to Real Transfer into an engineering problem with measurable gaps rather than an open question.
A clean split between reasoning and control. Gemini Robotics ER 2 (Jul 2026) is shipped as a high-level model that plans, tracks progress from video, and calls a separate action model. Teams can now improve planning and control independently instead of retraining one monolith.
Capital reaching the hardware layer. Apptronik closed a Series A totaling more than $935 million in February 2026, reported alongside a $5 billion valuation by CNBC. Sustained funding of this size is what puts enough identical platforms in the field for fleet-scale data collection to be possible at all.
None of this solved the interaction data problem. Simulation cannot originate contact dynamics, friction, or sensor noise that were never observed, and general models still fail at precise physical action: on EmbodiedBench (ICML 2025), the strongest evaluated model scored 28.9% on average across low-level tasks while performing well on high-level semantic ones. Capability at the top of the stack outran competence at the bottom.
Embodied AI Compared to Physical AI
| Dimension | Embodied AI | Physical AI |
|---|---|---|
| Primary question | How does an agent learn through acting? | How does learned intelligence run reliably on real machines? |
| Unit of analysis | The perception-action loop | The deployed system and its operating envelope |
| Typical setting | Simulators and research platforms as well as real robots | Production robots, vehicles, industrial equipment |
| Where the difficulty sits | Generalization and grounding | Reliability, latency, safety, and maintenance |
| Typical evaluation | Task success across held-out scenes and instructions | Closed-loop performance and failure rate under real conditions |
| Data center of gravity | Diverse demonstrations and interaction episodes | Synchronized production logs and outcome records |
The row that causes the rest is the unit of analysis. Once the object of study is the loop rather than the deployed system, simulation becomes legitimate, generalization becomes the headline metric, and diversity of data matters more than fidelity of any single record. Once the object of study is the deployed system, the reverse holds and a perfectly synchronized log from one production line is worth more than a thousand varied simulated episodes. Neither view is wrong. Most teams need both, which is why work that treats them as competing schools tends to produce policies that demo well and deploy badly.
Why Embodied AI Matters
Take one narrow task: a robot picks a mug from a counter and places it on a shelf. Stated in a sentence, it sounds like something a capable model should already handle. Enumerate what the training record actually has to contain and the difficulty becomes visible. The system needs synchronized RGB from at least two viewpoints, depth or another spatial signal, joint proprioception sampled at control rate, gripper aperture and commanded state, force or torque at the contact, the language instruction that defines the goal, and an outcome label saying whether the placement actually succeeded. Every one of those streams arrives from a different device with its own clock and its own sampling rate. See Multimodal Dataset Synchronization for why this alignment is the load-bearing step.
Now a specific failure. Suppose the wrist camera and the gripper telemetry are offset by a few tens of milliseconds and nobody validated the alignment. The frame that gets labeled as the moment of grasp is actually the frame before contact. Trained on thousands of such records, the policy learns to close the gripper slightly early, which in practice means it closes on air or on the rim and the mug tips. Nothing in the training loss reveals this, because the labels are internally consistent. The error is in the timebase, not the annotations.
A second failure comes from what the dataset omits. If demonstrations contain only clean successes, the policy never sees a mug slipping halfway through a lift, so it has no behavior for the state it lands in when one does. It will attempt the remainder of the nominal trajectory against a world that no longer matches. This is the recovery gap, and it is why Closed-Loop Policy Evaluation is not optional: an open-loop score computed against held-out demonstrations cannot express it, since the recovered state never appears in the held-out set either.
Both failures share a downstream consequence that makes them expensive. The policy passes lab evaluation and fails in deployment, and the deployment logs cannot say which of the two caused it, because they record what the robot did without recording the outcome, the failure type, or the alignment confidence of the streams they were derived from. The team then does the only thing available and collects more data, which reproduces the same defect at a larger scale. Data quantity is the visible constraint in embodied AI. Data trustworthiness is the one that actually decides whether a policy transfers.
Industry Perspective
Leading organizations take materially different approaches to Embodied AI. Google DeepMind develops a closed model stack around Gemini and provides embodied reasoning through its developer platform. Physical Intelligence releases model weights and inference code through openpi while pursuing heterogeneous co-training. The Open X-Embodiment consortium pools data from 34 participating institutions. Apptronik, Figure, and other robotics companies focus on humanoid platforms, while much of the research corpus still comes from tabletop arms and mobile manipulators. These programs differ in their approach to open versus closed weights, embodiment, and the role of simulation in training.
Despite these differences, their published work reveals several areas of convergence. Interaction data has become a central constraint alongside compute and architecture. Open X-Embodiment and π0.7 use heterogeneous corpora spanning multiple tasks, embodiments, and data sources. The Gemini Robotics ER line and π0.5 also distinguish high-level reasoning or subtask planning from low-level action execution. Across these programs, the training record is becoming richer than a simple observation and action pair, incorporating elements such as task metadata, subgoal representations, and more varied trajectory outcomes.
This convergence supports a narrower but still consequential conclusion. As robotics programs adopt heterogeneous and more richly structured corpora, the data operation becomes a durable asset alongside the models trained on it. Model families are iterating rapidly, while a corpus that is correctly synchronized, versioned, and honestly labeled can remain useful across multiple model generations.
Kotwel’s Role
Kotwel does not build robots, does not train foundation policies, and does not sell hardware. That exclusion is deliberate, because it keeps Kotwel complementary to every robotics program regardless of which embodiment or architecture a team has bet on. Kotwel works on the data layer underneath all of those bets.
Kotwel’s role sits in the data operation required to make these systems trainable and auditable. That operation begins with preserving the provenance, timing, and schema of records collected from cameras, teleoperation systems, wearables, and robot sensors. It then requires validating not only whether each record conforms to a schema, but whether the signals describe a physically plausible interaction. A gripper may report closure while force at the contact remains at baseline, a joint trajectory may imply an impossible velocity, or a sensor channel may flatline without triggering a formatting error. These records can pass conventional data validation while still teaching the wrong behavior.
The same discipline extends to versioning and quality control. Corrections should produce traceable dataset versions rather than silent in-place edits, so a policy regression can be connected to the data change that preceded it. Human review should concentrate on uncertain outcomes, automatically flagged trajectories, contact transitions, and recovery segments rather than treating every frame as equally risky. Kotwel contributes its experience in structured annotation, validation, quality control, and dataset delivery to this layer, while the robotics team retains responsibility for hardware-specific limits, control behavior, and policy evaluation.
Key Concepts
Physical AI is the deployment-side view of the same field, concerned with reliability on real machines.
Multimodal AI supplies the alignment machinery embodied agents depend on across vision, language, and sensor streams.
Vision-Language-Action Models map visual observations and instructions directly to robot actions.
Behavior Cloning is the supervised imitation baseline that most embodied policies still start from.
Diffusion Policy generates multi-step action sequences and handles the multi-modality that regression baselines average away.
Reinforcement Learning for Robotics supplies the correction signal when demonstrations alone cannot cover recovery.
World Models for Robotics let an agent predict consequences internally instead of discovering them through contact.
Sim to Real Transfer is the problem that decides how much simulated experience is worth.
Teleoperation Data is the dominant source of real manipulation demonstrations and the main cost driver.
Egocentric Data extends the corpus with human first-person video that no robot rig can match in volume.
Synthetic Data for Robotics amplifies an existing corpus but cannot originate contact dynamics never observed.
Multimodal Dataset Synchronization aligns sensor timebases and is the failure point behind mislabeled contact events.
Closed-Loop Policy Evaluation measures policies by running them, which is the only way recovery behavior becomes visible.
Data Flywheel describes the loop in which deployment failures become the next training corpus.
Further Reading
Primary sources
Open X-Embodiment: Robotic Learning Datasets and RT-X Models, Oct 2023. The pooled corpus of over 1 million trajectories across 22 embodiments.
π0.5: a Vision-Language-Action Model with Open-World Generalization, Apr 2025. Multi-stage manipulation in unseen homes.
π0.7: a Steerable Generalist Robotic Foundation Model, Apr 2026. Context conditioning and zero-shot cross-embodiment results.
Gemini Robotics 1.5, 25 Sep 2025. The reasoning and action split, with motion transfer across embodiments.
EmbodiedBench, ICML 2025. 1,128 tasks measuring the gap between high-level reasoning and low-level action.
Intelligence without representation, Brooks, Artificial Intelligence 47, 1991. The origin of the embodiment argument.
Data Scaling Laws in Imitation Learning for Robotic Manipulation, Oct 2024. How demonstration count, environment count, and object count trade off.
Industry and strategy
Gemini Robotics ER 2, Google DeepMind, Jul 2026. Embodied reasoning released for developer access.
Gemini Robotics-ER 1.6, Google DeepMind, 14 Apr 2026. Spatial reasoning, success detection, and instrument reading benchmarks.
openpi, Physical Intelligence, ongoing. Open weights and inference code for the π model family.
Apptronik closes over $935 million Series A, Feb 2026, with valuation reporting from CNBC, 11 Feb 2026.
Open X-Embodiment repository, Google DeepMind, ongoing. Dataset access and pretrained checkpoints.
At a glance
Key takeaways
Embodied AI is defined by the closed loop between action and observation, which makes its data sequential, consequential, and far more expensive to collect correctly than static datasets.
Every capability jump since 2022 traces back to enlarging and standardizing the data rather than to a new control method, as the pooled Open X-Embodiment corpus and the π and Gemini Robotics model families all show.
High-level embodied reasoning has outrun low-level physical competence: the strongest model evaluated on EmbodiedBench averaged 28.9% on low-level tasks while handling semantic ones well.
The failures that break deployed policies are usually data failures, such as a misaligned sensor timebase or a demonstration set containing only clean successes, and neither is visible in ordinary training metrics.
Organizations that disagree on embodiment, openness, and simulation still converge on heterogeneous, failure-inclusive corpora, which makes the data operation the asset that outlasts any single model.
FAQ
Is embodied AI just a new name for robotics?
No. Robotics is the engineering discipline of building and controlling machines, and much of it involves no learning at all. Embodied AI is the study of agents that acquire competence through interaction, and a substantial part of it happens in simulators with no physical machine involved. The overlap is large and growing, but a classically programmed industrial arm is robotics without embodied AI, and a Habitat agent is embodied AI without robotics.
How is embodied AI different from Physical AI?
They study the same field from opposite ends. Embodied AI asks how an agent learns through acting, and treats generalization as the headline metric. Physical AI asks how learned intelligence runs reliably on real machines, and treats failure rate under real conditions as the headline metric. A team working only on the first tends to produce policies that demo well and deploy badly.
Why can’t robots just learn from internet video?
Internet video shows what happened but not what was commanded, felt, or attempted. It contains no proprioception, no force at the contact, no gripper state, and no record of the failures that were edited out. Human video is genuinely useful as a source of visual and semantic prior, which is why Egocentric Data is part of most modern corpora, but it cannot supply the action labels a policy needs to learn control.
If simulation is cheap, why collect real data at all?
Simulation can only reproduce phenomena someone modeled. Contact dynamics, friction on a specific surface, cable stiffness, and the noise signature of a particular sensor are exactly the things that are hard to model and exactly the things that break policies. Simulation amplifies a real corpus efficiently. It cannot originate one.
What actually goes wrong most often in embodied AI datasets?
Timebase misalignment between streams, and demonstration sets that contain only successes. The first silently mislabels the moment of contact, so the policy learns the wrong trigger point. The second leaves the policy with no behavior for recovering from a partial failure. Both pass schema validation, both look fine in training curves, and both surface only when the policy runs in a real closed loop.
