The Data Bottleneck

8.1 Why robot data is not like internet text

The central asymmetry is that text and images are exhaust. Humans produce them anyway, for their own reasons, and the internet stores them for free. Robot data is not exhaust: every trajectory must be deliberately produced by a specific human operating a specific machine in a specific room, and it is worthless without synchronised action labels.

Ken Goldberg put a number on the gap in Science Robotics, 27 August 2025:

"The amount of internet-scale data (texts and images) used to train contemporary large vision language models is on the order of 100,000 years — it would take a human that long to read or view it."

The largest teleoperated robot dataset, he notes, is roughly one year of data. Extrapolating current collection rates, he concludes that a general-purpose robot trained on a ChatGPT-sized corpus of robot data "will be available in... 100,000 years." He then prices the alternatives and finds each wanting: YouTube holds roughly 35,000 years of video, but extracting precise 3D motion from 2D footage is unsolved; and simulation still fails on contact-rich manipulation, where submillimetre geometric inaccuracies destroy contact predictions.

The inventory bears him out. Against 100,000 years:

• Open X-Embodiment: 1M+ trajectories pooled from 34 labs.
• DROID: 350 hours, 50 collectors, 13 institutions, twelve months of work.
• AgiBot World: 2,976.4 hours — the largest purpose-built corpus in existence, requiring 100+ robots in a dedicated 4,000 m² facility.

Three thousand hours is a rounding error against a hundred thousand years. This single ratio explains almost every strategic decision described in Chapters 4 through 7.

8.2 Do scaling laws hold?

Partially, and with an important twist that reframes the whole problem.

Lin et al., "Data Scaling Laws in Imitation Learning for Robotic Manipulation" (2024), collected 40,000+ demonstrations and ran 15,000+ real rollouts to answer the question empirically. Their finding: generalisation follows a power law in the number of environments and objects, not in raw demonstration count. Beyond a threshold of demonstrations per environment, additional demos in that environment add almost nothing. Their headline: four collectors working a single afternoon — across many environments and objects — produced roughly 90% success on novel objects in unseen environments.

Physical Intelligence's π0.5 ablations report the same shape, with returns flattening around ~100 training homes.

This is the twist that matters: the bottleneck is diversity, not volume. A million trajectories of one robot doing one task in one room is worth less than ten thousand trajectories spread across a thousand rooms. It reframes the collection problem from "how do we get more hours" to "how do we get more situations," which is why in-the-wild interfaces like UMI and human-video pretraining are strategically important out of proportion to their raw hour counts.

8.3 The most rigorous negative control

TRI's "A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation" is the field's methodological high-water mark, and its conclusions are deliberately unflattering to its own programme.

Training on ~1,700 hours, with blind evaluators, 1,800 real-world trials, 50 rollouts per task-policy pair, and 27% of rollouts re-scored for quality control (which revealed a 2.31% success-rate discrepancy between graders), TRI found that pretraining genuinely helps: finetuned large behaviour models need "less than 30% of the data needed for training from scratch" in simulation, and on one real task 15% of the data sufficed.

But the gains are smooth and gradual, with no discontinuity at current scales. There is no emergent phase transition in sight. Whatever is coming, it is not arriving as a sudden capability jump at 10,000 hours.

That paper is as much an indictment of the field's evaluation hygiene as it is a scaling result. Before it, the modal published result was a video and a success rate from an unspecified number of author-scored trials. After it, that is no longer defensible.

8.4 The failure-data problem

Almost every dataset in Chapter 7 is a corpus of successes. Teleoperators do not deliberately produce failures; vendors are not paid for them; and papers do not report them. RoboMIND's 5,000 labelled failures are a conspicuous outlier.

This matters for three reasons. You cannot train a good reward model without negative examples — which is why Finn's RoboReward had to build its own benchmark. You cannot train a good uncertainty estimate, which is what Pavone's OOD-detection work and Roy's belief-consistency work both need. And you cannot train recovery behaviour, which is precisely what separates a 90% policy from a 99.9% one.

The companies with the best failure data are the deployed ones — Ambi, Dexterity, RightHand, Chef — because a production line records what went wrong whether you want it to or not. That is an underrated argument for the deployment-flywheel thesis.

---