Summary
The Waymo Open Motion Dataset is a public autonomous driving dataset from [[Waymo]] built for motion forecasting. It represents road users as time-indexed tracks beside static road geometry and changing traffic signal states, allowing researchers to study how vehicles, pedestrians, and cyclists may move together rather than treating perception as the benchmark task.
Overview
Waymo introduced the dataset with the 2021 paper Large Scale Interactive Motion Forecasting for Autonomous Driving. The original release described more than 100,000 scenes drawn from six United States cities and generated through scenario mining plus an offboard perception pipeline.
Waymo’s dataset catalog, last updated in October 2025, lists 103,354 segments, each 20 seconds long at 10 Hz, totaling 574 hours and more than 20 million frames. It reports 10.8 million tracked objects across vehicles, pedestrians, and cyclists, with data from San Francisco, Phoenix, Mountain View, Los Angeles, Detroit, and Seattle.
The Motion Dataset sits beside two separate resources under the Waymo Open Dataset name. The Perception Dataset contains high resolution sensor data and labels for 2,030 segments, while the Motion Dataset contains trajectories and corresponding 3D maps for 103,354 segments as of October 2025. Access is free for research under Waymo’s March 2025 license for noncommercial use.
What Waymo Open Motion Dataset Does
Each scenario organizes moving agents, the self-driving car, and the road context on a shared timeline. The public scenario schema stores each object’s 3D box center, box dimensions, heading, two-dimensional velocity, object type, tracking identifier, and a validity flag for every state. Timestamps align agent states with dynamic map states, while a current-time index separates observed history from future data to be predicted.
The same schema marks the autonomous vehicle track, objects of interest with detected interactive behavior, and tracks selected for prediction. Static map features and time-varying traffic signal states give a forecasting model context that a trajectory alone does not carry. Waymo’s October 2025 catalog says each segment includes 3D map data with lane centers, lane boundaries, road boundaries, crosswalks, speed bumps, stop signs, and driveway entrances.
The object tracks are not the output of a real-time onboard detector. Waymo states that the 3D boxes were produced by a model trained on the Perception Dataset. The associated Offboard 3D Object Detection paper describes using point cloud sequences, multi-frame detection, and object-centric refinement after a drive has finished. That relationship makes WOMD a concrete example of [[Offboard Auto-labeling]] output becoming benchmark ground truth.
The original paper defines forecasting tasks for both individual agents and interacting groups. Its release uses a 10 Hz sampling rate and an eight-second future horizon, and provides metrics for marginal predictions and joint predictions. In the paper’s dataset comparison, WOMD contains 7.64 million unique tracks with an average track length of 7.04 seconds, figures for the original 2021 release rather than the current catalog’s later object count.
Physical AI Relevance
Motion forecasting connects perception to planning. A detector can identify a vehicle at the present instant, but a planning system needs a distribution over where that vehicle, nearby pedestrians, and cyclists may move next. Interactions such as merges, unprotected turns, and crosswalk negotiations make those futures conditional on several agents rather than on one track in isolation.
WOMD matters because it exposes [[Trajectory Data]] together with road geometry and signal state at a scale designed for interaction research. Scenario mining changes the training distribution by selecting interesting behaviors instead of presenting only an unfiltered sample in which routine motion dominates. The original paper explicitly labels groups of interacting objects so models can be evaluated on joint forecasts as well as per-agent forecasts.
The dataset also separates motion research from perception benchmarking. Its core motion representation gives researchers tracked boxes and map context, which reduces the need to solve raw sensor detection before studying forecasting. Later WOMD variants add LiDAR or camera-derived features, but their presence should not be confused with the core trajectory benchmark or with the separate Perception Dataset.
Connections
Waymo Open Motion Dataset has a generated_by relationship with [[Offboard Auto-labeling]] because its tracked 3D boxes come from an offline perception system that can use a recorded sequence.
Waymo Open Motion Dataset has an is_a relationship with [[Trajectory Data]] because its central learning record is a timestamped sequence of agent states.
Waymo Open Motion Dataset has a generated_by relationship with [[Waymo]] because Waymo created and maintains the release within its broader open dataset program.
Waymo Open Motion Dataset also has a generated_by relationship with [[Scenario Mining]]. Waymo selected scenes for behaviors such as merges, lane changes, intersections, and unprotected turns rather than treating every logged interval as equally informative.
Limitations
The released distribution is intentionally mined for interesting interactions. That makes it useful for behavior prediction, but it does not represent an unbiased estimate of how frequently those events occur during ordinary driving. Results on WOMD therefore measure performance on its selected scenarios, not event prevalence across Waymo’s fleet or across all road environments.
Its geographic coverage is limited to six United States cities in the 2021 paper and the October 2025 catalog. Road conventions, agent behavior, weather, infrastructure, and traffic control patterns outside those locations are not established by the dataset.
The core benchmark depends on auto-labeled 3D tracks. The offboard pipeline was designed to reduce perception noise, but downstream users still inherit its object definitions, tracking decisions, and failure modes. A validity flag says that a state is valid or missing, while the public schema does not encode a reason such as occlusion, range, or a track association failure. Those cases can have different implications for sequence filtering and model supervision.
The dataset is also a research release rather than evidence of the full Waymo Driver. Waymo states that the public data is only a fraction of its training data and should not be used to evaluate the real-world performance of any vehicle.
Links
Kotwel’s Role
Kotwel is the data-quality and data-operations partner for Physical AI. We turn multimodal physical-world and synthetic data into synchronized, structured, validated, and traceable datasets for training, evaluation, and continuous improvement. We make consequential data choices explicit and develop measurable quality criteria that enable organizations to benchmark and improve the data behind physical AI systems.
Scope and responsibility
Kotwel works at the data layer and remains independent of hardware, sensing architecture, and model family. We define data-quality criteria, audit datasets, and assess conformance against documented requirements. Clients retain control over program-specific conventions, intended model behavior, and operating requirements, provided those choices are explicitly documented and consistently applied. We do not build robots or autonomy stacks, develop control systems, or train foundation models. Decisions about system safety, legal compliance, and deployment remain with the client and its qualified advisers.
The validity field creates a concrete specification decision when a track state is missing. The flag records only that a state is valid or missing; the public schema does not encode whether the cause was occlusion, range, or a track association failure, and those cases can have different implications for sequence filtering and model supervision. A rule drawn from the dataset alone therefore cannot depend on the cause, and it involves two separate decisions.
The first is at the level of a single state: whether a missing state is excluded from the treatment or preserved explicitly as an unknown observation. The distinction between them is whether the missingness remains represented and traceable in the record or is excluded from it.
The second is at the level of the track: when missing states form a gap, whether the track is treated as continuous across it or broken. Treating it as continuous maintains continuity across the gap, but assumes continuity when the flag cannot show whether the gap came from occlusion, range, or an association failure. Breaking the track avoids making that assumption, but gives up continuity across the gap. If a gap length decides where continuity breaks, that threshold is a substantive choice for the program and model owner. Kotwel surfaces both decisions in the labeling and validation specification, makes the agreed treatment and threshold explicit, applies the agreed treatment consistently across sequences, checks conformance, and keeps the rule and its application traceable. It does not decide where a gap becomes a break, or a forecasting model’s target behavior.
At a glance
Key takeaways
WOMD supports motion forecasting with agent tracks, road geometry, and changing traffic signal states.
Its core 3D tracks are auto-labeled through an offboard perception pipeline, so forecasting users inherit that pipeline’s tracking decisions and failure modes.
Scenario mining emphasizes interesting interactions; benchmark results do not establish how often those events occur in ordinary driving.
Geographic coverage is limited to six United States cities, and the research release does not establish the real-world performance of the Waymo Driver.
Missing-state validity flags do not explain why a state is absent; sequence filtering and continuity rules need explicit decisions.
