Robots Need Brains Built for the Physical World, Not Just Tuned From Video

If you walked the main hall at this year's WAIC, you couldn't miss it: embodied intelligence was everywhere. More than 200 companies packed into the H3 pavilion, showing off robots that sort parcels, patrol corridors, and tidy shelves. The shift from showy demos to practical tasks was obvious. But the real story isn't the hardware. It's the invisible part that decides whether a robot can actually work in the real world: the brain.

The Data Problem: No Standards, No Consensus

Shen Yujun, chief scientist at Ant Lingbo, has been thinking about this problem for a while. His starting point is blunt: taking a video model built for the digital world and fine-tuning it for robots is a shortcut, not a solution. It's path dependence. And he should know—his background is in generative models.

The trouble begins with data. There is no agreed-upon format for robot training data. One company might think a single head camera is enough. Another insists on five-finger grippers. Some use teleoperation. Even within head-mounted setups, there are variations: two cameras for vision and depth, or extra cameras to track the head's position. The question of whether you need just vision or also positional info is still open. The same goes for hands. Do you need to see the skeleton through the head camera, or do you need hand pose data, or touch? Without a clear answer, mass data collection is stuck.

Why hasn't the industry converged? Because the model architecture hasn't converged either. Nobody knows exactly which modalities the brain needs. And even if you pick a modality, the required precision is up for debate. Shen gives an example: hand pose. Some want millimeter accuracy, others are fine with centimeters. Higher precision usually means higher cost, which hurts scale. So there's a tradeoff between data quality and cost, and no one has figured out the right balance.

There's also a disconnect between the teams. Data and sensor folks have their own metrics. Model builders care about different things. If a dataset has ten dimensions, the model might only rely on five, but the model team can't say which five. So data teams try to improve everything, which drives costs through the roof. It's a chicken-and-egg problem, and it's why data collection is still in the exploratory phase.

What Data Matters? Real, Not Just Simulated

Lingbo's approach is to focus on real data, but not all real data is equal. The first type is teleoperation data from actual robots. No matter how good models get, you can't avoid this. A robot has to know its own body, and the best way is to collect data from that body. This data doesn't need to be huge, but it has to be very high quality.

The second type is real, but not from a robot—like first-person human data or data from grippers. This is precious because it captures how humans actually do things. In simulation, robots move differently than people do. Humans have instincts, subtle motions that are hard to fake. So this human-centric data is key, and it's easier to scale than teleop data.

As for simulation, Shen is cautious. For a specific, well-defined scenario like autonomous driving, simulation makes sense. You can build a dedicated physics engine and train for that task. But Lingbo is aiming for general-purpose robots. If you want a robot to do a hundred different things, building a hundred simulators is too heavy. And simulation still can't replicate the human touch. When you open a bottle of water, you do it in a certain way—your hand knows what to do. Simulators can't capture that. So for pretraining, human data is more valuable.

When will the data flywheel spin? Shen is optimistic, but not about scaling laws. The model routes are slowly converging, and that helps data standards follow. Once models get better, they'll ask for more specific data, which will scale up, which will improve models further. It's a virtuous cycle, but he won't predict whether it's exponential or logarithmic. No inflection point yet, but the trend is good.

Why So Many Models? It's Not Confusion, It's Decomposition

Lingbo recently released a "full-stack brain 2.0" with six models, up from four in January. Some people wonder if that's just fragmentation. Shen says no. Each model solves a specific technical problem. The robot brain is a big puzzle, and if you try to build the whole thing at once and fail, you won't know which piece broke. So you test pieces individually.

True end-to-end control is the goal, but it's far off. It's not about whether to separate layers; it's about whether each modality is well understood. Vision has to learn its own features before you can fuse it with touch. The same goes for fusion, efficiency, and action execution. These are distinct challenges, and until they're solved, you can't assemble the final model. Shen compares it to reading a book: it gets thicker before it gets thinner.

VLA vs. VA: Both, for Now

One of the big debates in embodied AI is VLA (vision-language-action) versus VA (vision-action, or world models). Shen thinks neither is the final answer. VLA is good at alignment and language understanding. It's fast and robust to messy data. VA, on the other hand, handles randomness better—like predicting where a ball will land in a tabletop game. But VA requires cleaner data and is weaker at following changing instructions.

So why do both? Because they test different capabilities. VLA lets you validate data collection methods. VA explores prediction and dynamic modeling. Until a better architecture comes along, you need both to understand what works. And you can't just wait for the perfect fusion; you have to make progress with what you have.

What Does "Embodied-Native" Really Mean?

Lingbo's big idea is "embodied-native" models—designed from the ground up for physical reality, not adapted from digital ones. That means data that includes distance, touch, even temperature—modalities the internet doesn't have. It means model capabilities that prioritize speed and causality over beauty. In the physical world, time moves one way. You can't see the future. So models need causal attention, not the bidirectional attention common in video generation.

Training such a model from scratch is hard. Shen points to their vision model, LingBot-Vision, which builds on DINO but adds custom objectives for spatial perception. Getting that to scale wasn't trivial. The same goes for Mixture-of-Experts. They tried 10 experts and found only two were working. Balancing them took new strategies. And causal modeling? No one in the digital world has figured out the right recipe. It's all trial and error.

But Shen insists this is the only way. If you don't train from random weights, you don't really have a native model. You're just borrowing someone else's brain.

Safety Has to Be Baked In, Not Bolted On

Safety is another area where Shen thinks the industry is taking shortcuts. Most systems use "fence" safety: tell the robot not to get too close to a glass, so it stops. But you can't enumerate every dangerous situation. A robot pouring water is fine—unless there's an outlet nearby. The environment changes what's safe.

What we need is "native safety," a higher-order intelligence that lets the robot understand danger the way humans do. Humans have instincts to avoid harm. Robots don't. We need to build that in from the pretraining stage, not as an afterthought. It's a long road, but we have to start now if we want robots to ever be trustworthy in homes.

The ChatGPT Moment for Robots

When will robots have their ChatGPT moment? Shen thinks it won't be when a robot does a backflip. It'll be when ordinary people can participate. Just like ChatGPT let anyone chat with an AI, robots need a way for everyone to contribute—maybe by spending an hour a day generating training data while going about their lives. That's when robots become truly useful and ubiquitous. Until then, we're just watching demos.

At this year's WAIC, the robots were more capable than ever. But the real breakthroughs will come from the invisible work: building brains that understand the physical world from the ground up. That's the deep-water race, and it's just getting started.