Physical AI
How learned intelligence moves into physical action, why state transitions make robot data difficult, and how the field shifted robot behavior out of engineered code and into datasets and learned parameters.
Summary
An inertial measurement unit, or IMU, is a sensor assembly that measures linear acceleration and angular velocity. In the six-axis configuration used throughout this article, it combines a three-axis accelerometer with a three-axis gyroscope (Wang and Olson, 2015). It supplies the fast motion signal that cameras and other slower sensors use for pose estimation, stabilization, and Sensor Fusion, but its measurements contain bias and noise that accumulate when integrated.
Overview
An IMU is a hardware category rather than a single product or standard. Here, six-axis means three accelerometer axes plus three gyroscope axes, consistent with the two three-axis sensors described by Wang and Olson (2015). In common nine-axis product terminology, three magnetometer axes are added; the magnetometer supplies a magnetic-field reference whose usefulness depends on the local magnetic environment.
The accelerometer reports specific force, commonly expressed in meters per second squared. When a stationary sensor is supported against gravity, its output includes the support force, so interpreting the reading as motion requires an attitude estimate and a gravity model. The gyroscope reports angular velocity, commonly in radians per second or degrees per second. Integrating that rate produces an orientation change, while integrating acceleration after gravity compensation can produce velocity and position estimates.
The term covers devices ranging from low-cost microelectromechanical systems, or MEMS, to higher-performance inertial sensors. The University of Michigan’s multi-gyro study describes compact MEMS accelerometers and gyroscopes on the same chip and notes their use in aerial and ground robotics (Wang and Olson, 2015).
What an IMU Does
A synchronized six-axis recording represents two three-dimensional vectors at a measurement time: linear acceleration and angular velocity. Device interfaces may publish the two channels with separate timing or availability semantics, so the recording schema still has to document how they were acquired. A system may preserve those raw or calibrated samples, or fuse them into orientation, velocity, and pose estimates. The distinction matters because an estimated pose is a derived model output, not another direct sensor channel.
Sampling rates are often much higher than camera frame rates because inertial measurements capture motion between images and support low-latency state estimation. Project Aria Gen 1 provides a concrete published example: its two IMUs operate at 1,000 Hz and 800 Hz, while its scene and RGB cameras have maximum rates of 30 frames per second. The sensors are timestamped on a common clock at nanosecond resolution (Project Aria Gen 1 hardware specifications). Calculated from those published maxima, one second can contain 1,800 inertial samples across the two units but at most 90 frames across the three scene and RGB cameras (Project Aria Gen 1 hardware specifications). For this configuration, a data pipeline has to preserve timestamps and sampling identity rather than force a one-to-one row match.
An IMU output is shaped by more than motion. Bias is an offset between the reported and true value. Noise density describes noise relative to bandwidth, while angle random walk and velocity random walk describe how integrated gyroscope and accelerometer noise spreads with time. Scale-factor error changes the gain, axis misalignment mixes motion between axes, and temperature can change bias and scale. VectorNav’s specification guide distinguishes in-run bias stability, turn-on repeatability, temperature sensitivity, random walk, and scale-factor error as separate properties rather than one generic accuracy number (VectorNav IMU specifications).
Calibration may include per-axis bias, scale, nonorthogonality, and the rigid transform between the IMU and another sensor. Factory calibration cannot remove every error that changes after startup or with temperature. Some systems estimate parameters during operation. Project Aria’s Machine Perception Services, for example, estimates time-varying camera and IMU intrinsics and extrinsics at the scene-camera frequency (Project Aria MPS documentation).
Physical AI Relevance
A robot or wearable that estimates pose from visual and inertial data uses the inertial stream during intervals when visual evidence is sparse, blurred, occluded, or geometrically ambiguous. IMU samples constrain short-term rotation and acceleration, while cameras, LiDAR, wheel odometry, or positioning systems provide references that can limit accumulated drift. In visual-inertial odometry, these streams jointly estimate trajectory. In control systems, inertial readings can also support stabilization and rapid disturbance detection.
For Physical AI datasets, the IMU is important as both evidence and context. It can record head motion in Egocentric Data, body-segment motion in Whole-Body Teleoperation, and platform dynamics in mobile robots. Its high rate makes small timing errors consequential: an inertial sample associated with the wrong image can describe a different point in a fast turn. That makes Multimodal Dataset Synchronization and Robot Calibration and Coordinate Frames part of the data contract, not postprocessing details.
The raw stream can also help quality assurance. Stationary intervals can expose nonzero gyro output or an accelerometer norm inconsistent with the expected gravity model. Sudden timestamp gaps, repeated samples, clipping at the measurement range, or disagreement between redundant units are detectable from the recording. These checks reveal symptoms. They do not, by themselves, establish the true motion.
Connections
IMU generated_by no broader data node in this graph because it is itself the sensor source.
Project Aria uses IMU to record head motion and to support its visual-inertial trajectory products.
IMU enables Sensor Fusion by providing high-rate motion measurements between observations from slower or externally referenced sensors.
IMU related Multimodal Dataset Synchronization because its samples must retain clock, rate, and timestamp semantics when aligned with images or actions.
IMU related Robot Calibration and Coordinate Frames because an IMU-to-camera or IMU-to-robot transform determines how its vectors are interpreted in another frame.
IMU related Whole-Body Teleoperation when wearable inertial sensors record the motion of multiple body segments.
Limitations
An IMU does not directly measure position. Position inferred from acceleration requires gravity compensation and repeated integration, so bias and noise accumulate. The Michigan study models gyroscope output as angular rate plus bias and white noise, with the bias following a random walk. Under that model, both angle random walk and rate random walk produce unbounded orientation error, and rate random walk dominates over long integration periods for the MEMS devices examined (Wang and Olson, 2015). The exact drift rate is device-specific and operating-condition-specific, so a universal time-to-failure figure would be misleading.
A magnetometer can constrain heading relative to the local magnetic field, but a nine-axis label does not guarantee a reliable global orientation. Local magnetic disturbances and calibration quality affect that reference. Likewise, a high sample rate does not guarantee accurate timing. The capture system still has to document which clock produced each timestamp, whether timestamps refer to sampling or arrival, and whether dropped samples are represented.
Post-hoc inspection is limited. A reviewer can see that a camera frame is blurred or points at the wrong scene. An inertial trace has no equally direct visual interpretation. Plausibility checks can detect clipping, discontinuities, implausible stationary behavior, or inconsistency with another stream, but separating true motion from bias requires an external reference, a trusted motion constraint, or a calibrated redundant sensor. If the recording lacks those references, the dataset should preserve uncertainty rather than converting a plausible trace into asserted ground truth.
Links
Kotwel’s Role
When an inertial stream disagrees with video around an event boundary, two readings may both be defensible: the IMU may have captured real motion before it became visible, or its clock, frame transform, bias, or calibration may be wrong. The chosen rule changes which timestamp and motion state the dataset treats as the event. Kotwel surfaces that decision in the specification, checks the observable timing and consistency evidence across streams, and leaves sensor-state estimation to the robotics team rather than labeling an inferred trajectory as measured truth.
At a glance
Key takeaways
In the terminology used here, a six-axis IMU measures three-axis acceleration and three-axis angular velocity; common nine-axis devices add three magnetometer axes (Wang and Olson, 2015).
In multimodal recordings such as Project Aria, inertial sampling is much faster than camera capture, so timestamps, clock provenance, and calibration versions belong in the dataset schema (Project Aria Gen 1 hardware specifications).
Bias, noise, scale error, temperature effects, and axis misalignment prevent raw inertial integration from serving as durable ground truth.
Dataset quality checks can find gaps, clipping, implausible stationary behavior, and cross-sensor inconsistency, but they cannot reconstruct missing external truth.
