Robotics
Where the seven-decade-old engineering discipline ends and trained behavior begins, why 4.7 million installed machines still run on specified motion, and what breaks when a robot has to handle variation nobody wrote down.
Physical AI
How learned intelligence moves into physical action, why state transitions make robot data difficult, and how the field shifted robot behavior out of engineered code and into datasets and learned parameters.
Summary
Physical AI describes intelligent systems that perceive, reason, and act through machines in the physical world. It connects learned models with sensors, control systems, and actuators in robots, vehicles, industrial equipment, and other autonomous machines. Unlike digital AI, Physical AI produces actions that alter the state from which the system must make its next decision. Its central challenge is therefore not model capability alone, but whether the data preserves the timing, context, and physical outcomes required for reliable action, which is the layer this note follows most closely.
Physical AI: Definition and Scope
Physical AI is the application of artificial intelligence to systems whose decisions produce physical actions. A complete system combines three functional layers. Perception captures the world through cameras and other sensors. Reasoning interprets those observations and selects or plans an action. Actuation executes the action through motors, grippers, wheels, or other mechanisms.
Physical AI differs less by replacing the classical robotics stack than by increasing the role of learned components within it. Perception, reasoning, policy generation, or several of these functions may be learned from data. Motion planning, state estimation, control theory, and safety mechanisms continue to provide essential structure in many deployed systems. Robotics is the engineering discipline; Physical AI is a current direction within it.
The term overlaps with Embodied AI, but the emphasis differs. Embodied AI is the broader academic framing that intelligence develops through an agent’s body and interaction with an environment. Physical AI is commonly used as the industry and engineering frame for deploying learned intelligence in machines. At CES on 6 January 2025, NVIDIA CEO Jensen Huang presented physical AI as the next stage after perception and generative AI, describing systems that reason, plan, and act in the physical world. This helped establish the term in current industry usage through NVIDIA’s robotics, autonomous vehicle, and industrial AI platforms.
Physical AI covers robot learning, autonomous vehicles, industrial automation, and drones. It does not cover software that produces only digital outputs unless those outputs participate in a physical perception and action loop.
How the Field Arrived Here
| Period | Milestone | Significance |
|---|---|---|
| 1961 | Unimate begins factory work at General Motors | Industrial robots demonstrate repeatable physical automation through programmed motion. |
| 1986 | Rodney Brooks publishes the subsumption architecture | Intelligence is framed as emerging from interaction with an environment rather than only from an internal world model. |
| 1990s to 2000s | Probabilistic robotics and SLAM mature | Statistical state estimation and mapping improve operation under sensor uncertainty, while most skills remain engineered. |
| 2016 to 2018 | Guided policy search and QT-Opt | Manipulation policies are increasingly learned from physical experience rather than specified entirely by engineers. |
| December 2022 | RT-1 | A transformer policy is trained across many real robot tasks using a shared action representation. |
| July 2023 | RT-2 | A Vision-Language-Action model transfers semantic knowledge from web pretraining into robot control. |
| October 2023 | Open X-Embodiment | A collaboration across 21 institutions and 22 robots assembles data covering 527 skills and 160,266 tasks. |
| June 2024 | OpenVLA | An open 7B parameter VLA trained on 970,000 robot demonstrations broadens access to adaptation research. |
| March 2025 | GR00T N1 and Gemini Robotics | Foundation model methods reach humanoid control, with an open humanoid model and an embodied reasoning system released in the same quarter. |
| June 2026 | Cosmos 3 | NVIDIA releases an open omnimodal world model that combines physical reasoning, world generation, and action generation. |
| July 2026 | Isaac GR00T N1.7 | NVIDIA documents an updated reasoning backbone, task decomposition, policy evaluation workflow, and deployment support for humanoid policies. |
The pattern is a steady increase in the share of robot behavior represented in datasets and learned parameters. That change shifts more of the engineering burden toward sourcing, aligning, evaluating, and maintaining data, without eliminating the classical robotics systems around the learned policy.
Why Now
Physical AI did not become viable through a single breakthrough. Several capabilities matured at roughly the same time.
Foundation models supplied reusable priors. A robot no longer has to learn every object concept from the same small interaction dataset used to teach control. RT-2 showed that vision and language knowledge acquired through web pretraining could support robot action, allowing robot data to concentrate more heavily on physical interaction.
A shared model interface emerged. gave many teams a common formulation for conditioning actions on visual observations and language. They do not eliminate planning, control, or safety layers in every deployment, but they make more of the perception to action pipeline jointly trainable.
Collection hardware became more reproducible. ALOHA published an accessible bimanual teleoperation design, while ALOHA 2 improved robustness and ergonomics. Open designs and commodity components let more labs reproduce demonstration collection workflows, although operator time, maintenance, and environment setup still constrain throughput.
Simulation and generative world models became more useful as data infrastructure. Systems such as Cosmos 3 can broaden scenario coverage and support training or evaluation. Their physical consistency remains model and task dependent. Contact dynamics, sensor behavior, calibration effects, and hardware specific failures still require real world validation.
Pooled datasets demonstrated cross-robot transfer. Open X-Embodiment showed positive transfer across robot platforms, making shared collection and standardized representations economically meaningful.
These developments expanded access, scale, and reuse. They did not remove synchronization errors, incomplete coverage, inconsistent action semantics, hardware constraints, safety requirements, or the gap between offline scores and closed-loop behavior.
Physical AI Compared to Digital AI
| Dimension | Digital AI | Physical AI |
|---|---|---|
| Typical prediction target | A token, pixel, score, or digital operation | An action that affects an environment |
| Data source | Existing digital corpora are often reusable | Task grounded interaction data must often be collected or simulated |
| Marginal cost of new experience | Usually low after the source exists | Bounded by operator time, hardware time, and environment access |
| Modality | Streams can sometimes be processed independently | Vision, depth, state, force, action, and time often need alignment |
| Evaluation | Fixed benchmarks support repeatable scoring | Closed-loop results depend on hardware and environment |
| Failure cost | Many outputs can be regenerated or rolled back | Failure can damage hardware, goods, or people |
| State transition | Output often leaves the evaluated world unchanged | Each action changes the state used by the next decision |
The physical state transition is the row that propagates into the others. Once an action changes the world, later observations depend on it. Sequence integrity, timing, and outcome labels therefore become load bearing properties of the training sample rather than secondary metadata. This distinction is typical rather than universal, since some digital systems also trigger irreversible operations.
Why Physical AI Matters
Task grounded robot trajectories containing synchronized observations, states, actions, and outcomes do not exist at web scale and generally cannot be obtained through conventional scraping. Consider a robot learning to pick up a cup and place it in a tray. A useful demonstration may require, at each timestep:
RGB video from one or more viewpoints
Depth or point cloud data
Joint positions across every degree of freedom
End effector pose
Gripper force or tactile readings
The natural language task instruction
A shared and traceable time base
A success or failure label, ideally with the reason for failure
The useful training unit is the synchronized trajectory, not any single frame. The main challenge is preserving the relationships among these streams.
Take a common failure. The camera stream is shifted by several frames relative to the gripper command, so training associates closing the gripper with the wrong visual moment. The files still appear complete, the training loss still declines, and a frame by frame review finds no obvious defect. During closed-loop execution the policy then closes too early or too late. A dropped frame teaches the wrong offset between visual state and action. Labels that vary across operators teach the model that the same scene warrants different behavior.
These are silent corruptions rather than cosmetic annotation defects, so quality control has to operate at the trajectory level rather than the frame level. The set of checks that does this forms a trajectory integrity gate: monotonic timestamp validation, sampling frequency checks, dropped frame detection, cross-stream offset estimation, comparison between commanded actions and observed state transitions, and replay of complete trajectories against their outcome labels. The gate is a decision, not a score. A trajectory that passes timestamp validation but fails replay is quarantined rather than shipped, then either repaired by recovering the correct offset or rejected when the underlying capture is unrecoverable. Because these failures are invisible to loss curves and to frame by frame inspection, a meaningful share of a raw collection batch is caught here rather than in training, which is why the gate sits before the training run rather than after it.
The International Federation of Robotics reported on 25 September 2025 that the worldwide operational stock of industrial robots reached 4,663,698 units in 2024, an increase of 9 percent from the previous year. Most were not general purpose learned robots, but the installed base shows the scale at which physical reliability already matters.
Investment has also accelerated. Crunchbase reported on 22 June 2026 that robotics startups had raised $18.8 billion during 2026 through that date, compared with $15 billion across all of 2025. These figures cover robotics startups broadly and do not show that every robotics program is adequately funded. They do show that increased investment has not removed the throughput constraint. New robot experience still consumes hardware time, operator time, controlled environments, and subsequent quality assurance.
For a team deciding whether to build this data capability or source it, the cost and risk drivers are concrete. Cost is set by operator time, hardware time, and controlled environment access rather than by storage or compute, so a dataset is expensive in the same places it is easy to corrupt. Risk concentrates in the offline to closed-loop gap, where data that scores well on fixed benchmarks can still degrade on real hardware. Before trusting a dataset, the questions worth asking are whether every trajectory carries a shared time base and an outcome label, whether action semantics stay consistent across operators and sessions, and whether provenance and version history are complete enough to audit when a policy later fails.
Industry Perspective
Leading organizations disagree substantially about architecture and deployment. NVIDIA’s GR00T line pairs a vision and language reasoning component with a high frequency action policy. Google DeepMind extends multimodal models with embodied reasoning. Physical Intelligence develops general purpose policies from diverse demonstrations. OpenVLA emphasizes open checkpoints and efficient adaptation. These are materially different approaches to how a robot should represent and generate action.
Despite those differences, their investments converge on one operational priority: expanding, standardizing, or generating the data used to train and evaluate physical systems.
Google DeepMind helped organize Open X-Embodiment to pool data across institutions and robot platforms.
NVIDIA developed Cosmos to support physical reasoning, simulation, synthetic data generation, and policy development.
Hugging Face develops LeRobot as shared tooling with reusable dataset formats and robot learning workflows.
Physical Intelligence reports training general purpose policies on diverse robot data in its π0 technical report.
These actions do not prove that data is the only binding constraint. Compute, hardware reliability, inference latency, safety, control frequency, and deployment economics remain material. They do show that improvements in model architecture have not removed the need for reusable demonstrations, calibrated sensor streams, consistent action schemas, and traceable evaluation records. The convergence is operational rather than architectural.
Kotwel’s Role
Kotwel does not build robots or select control architectures. That boundary is deliberate because reliable dataset operations are required across programs using different hardware and model families.
Kotwel focuses on multimodal robot data operations: designing collection protocols, structuring synchronized video, depth, force, and trajectory records, defining annotation instructions and consistent action schemas, running quality assurance before training, and delivering versioned datasets with documented provenance.
Key Concepts
Embodied AI is the academic framing of intelligence that develops through perception, action, and feedback in a body situated in an environment.
Robotics is the engineering discipline of designing and operating physical machines, including the mechanical, control, safety, and learned components around a policy.
Multimodal AI reasons across several data types. Physical AI systems commonly combine vision, depth, touch, proprioception, and language.
Autonomous Systems operate with limited direct intervention in dynamic environments.
Spatial AI represents three dimensional space and physical relationships required for navigation and manipulation.
Vision-Language-Action Models condition robot actions on visual observations and language instructions.
Closed-Loop Policy Evaluation tests a policy through repeated perception, action, and feedback on real hardware or a sufficiently representative environment, revealing compounding errors and state-dependent failures that offline metrics can miss.
Robot Demonstration Data records tasks performed through teleoperation or other teaching methods in a robot’s observation and action space.
Multimodal Dataset Synchronization aligns sensor, state, action, and annotation streams on a shared time base.
Synthetic Data for Robotics uses simulated or generated experience to expand scenario coverage beyond physical collection.
World Models for Robotics learn representations of physical state and change for prediction, generation, training, or evaluation.
Sim to Real Transfer studies how capability learned in simulation carries to physical hardware and where the reality gap causes failure.
Human Video Data provides broad behavioral evidence but usually lacks direct robot state and action labels.
Data Flywheel is the loop in which deployment events become reviewed data for future training and evaluation.
Further Reading
Primary sources
Open X-Embodiment: Robotic Learning Datasets and RT-X Models, 2023.
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control, 2023.
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots, 2025.
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware, 2023.
World Robotics 2025, International Federation of Robotics.
RT-1: Robotics Transformer for Real-World Control at Scale, 2022
LeRobot: State-of-the-art Machine Learning for Real-World Robotics, Hugging Face, 2024
π0: A Vision-Language-Action Flow Model for General Robot Control, Physical Intelligence, 2024
Industry and strategy
NVIDIA Launches Cosmos 3, June 2026.
Develop Humanoid Robot Policies End-to-End with NVIDIA Isaac GR00T, July 2026.
Robotics Startups On Fire As Venture Funding Surges, June 2026.
Scaling up learning across many different robot types, Google DeepMind, 2023.
At a glance
Key takeaways
Physical AI increases the role of learned perception, reasoning, and action within a broader robotics stack; it does not make planning, control, or safety engineering disappear.
A physical action changes the state used by the next decision, which makes trajectory timing, sequence integrity, and outcome labels central properties of training data.
Foundation model priors, shared VLA interfaces, reproducible collection hardware, generative world models, and pooled datasets expanded the field without removing its hardware, safety, data, and evaluation constraints.
Leading organizations use different architectures but invest in expanding, standardizing, or generating robot data, showing that data infrastructure remains an operational priority rather than proving that data is the only bottleneck.
Reliable Physical AI data requires a trajectory integrity gate and traceable dataset operations from collection through delivery.
FAQ
What is the difference between Physical AI, Embodied AI, and robotics?
Robotics is the engineering discipline, Embodied AI is the research framing, and Physical AI is the industry frame for deploying learned models in machines. Robotics covers mechanical design, electronics, state estimation, planning, control, and safety, and it exists whether or not anything is learned. Embodied AI studies how intelligence develops through a body acting in an environment, which is a claim about how learning happens. Physical AI describes the current engineering direction in which learned components take over more of the perception, reasoning, and action pipeline, inside a robotics stack that still needs planning, control, and safety around them.
If foundation models keep improving, will the data problem solve itself?
Better foundation models reduce how much data a robot needs in order to learn concepts, but they do not reduce the need for physical interaction data, because no model can pretrain on contact forces, joint trajectories, and outcomes that were never recorded. Web pretraining supplies priors about objects and language, which is why RT-2 transferred semantic knowledge into robot control. It does not supply the timing between a gripper command and the visual moment that command acted on. Architecture improvements change what share of a robot’s competence has to come from robot data, not whether that data has to be correct.
Can synthetic data replace real robot data?
Synthetic data cannot fully replace real robot data. Simulation and generative world models expand scenario coverage cheaply and can produce rare events that would be unsafe or impractical to stage physically, which makes them strong for breadth. Real hardware remains necessary to measure contact dynamics, sensor noise, calibration effects, and closed-loop behavior, because those are precisely the properties a simulator approximates. The working pattern is synthetic data for coverage and real data for validation, rather than one substituting for the other.
How many demonstrations does a robot task actually need?
There is no universal number because the requirement depends on task complexity, environmental variation, hardware, and the policy’s expected generality. ACT demonstrated individual tasks using 50 demonstrations per task, approximately 10 minutes of data, and achieved success rates of 80 to 90 percent as reported in 2023. At a broader scale, RT-1 used more than 130,000 episodes covering more than 700 tasks, collected with 13 robots over 17 months as reported in 2022. The practical question is therefore not only how many demonstrations a task needs, but whether the dataset covers the objects, conditions, failures, and recovery cases the robot will encounter.
Is it cheaper to collect robot data in house or to work with a data partner?
The cheaper operating model depends on collection volume, task complexity, available hardware, internal expertise, and the quality controls the program requires. A small team with an established rig and a narrow task may find direct collection practical because it avoids external onboarding and coordination costs. As collection expands across operators, sessions, environments, or hardware configurations, synchronization, review, action consistency, versioning, and provenance become substantial operating requirements. Teams should therefore compare the full cost of producing an auditable training dataset, not only the hourly cost of operating the teleoperation rig.
What makes a Physical AI dataset reliable?
A reliable Physical AI dataset preserves synchronization, calibration context, action semantics, outcome labels, provenance, and version history across every trajectory. It also has to cover the conditions the robot will meet at deployment, including failures and recoveries, not only clean successful runs. Reliability is verified at the trajectory level rather than the frame level, because the corruptions that matter, such as a shifted camera stream or an inconsistent action label, leave every individual frame looking correct.
