Si₀

Our recent work introduced Fe₀, an embodied foundation model built with a rigorously controlled setup to examine different levels of cross-embodiment transferability. This post goes under the hood to introduce Si₀ — the multimodal backbone driving the spatial and physical understanding behind those embodied efforts.

The development of Si₀ stems from an emerging consensus within the community: building an effective embodied foundation model requires a fundamental leap in spatial perception. This imperative directly intersects with a major direction in multimodal research: spatial intelligence — the push to make models that don't just caption an image but actively understand its 3D layout, objects, motion, and viewpoint. After all, a humanoid has to act in the physical world, not describe a picture of it, and a deeper grasp of space provides a stronger foundation for embodied control.

Acting on that intuition, we built Si₀ with enhanced physical-world understanding, pretrained on broad general-purpose data with a deliberate emphasis on real-world egocentric-exocentric data and robot-domain data, chosen to strengthen both its spatial understanding — where things are, how a scene is laid out, what is moving — and its embodied understanding — what a task affords and what to do next.

However, naively mixing data is a blunt instrument. It proves that spatial intelligence matters, but masks which component does the heavy lifting. Spatial intelligence is not a monolithic skill; it is a bundle of distinct capabilities. To build a principled foundation for embodied AI, we must move beyond the “alchemy” of data mixing. This post turns on a fundamental question: What exactly makes a good spatial-intelligence backbone for physical action?

We organize our answer around two axes that turn out to be orthogonal:

• Cognitive skills (§1–§2). The backbone must have the spatial-intelligence skills that acting requires — knowing precisely what and where, how a scene is laid out in 3D, what is moving, and what to do next — and we ablate which of them actually transfer to action.
• Visual distribution (§3). The backbone must also see the world the way the robot's own egocentric sensors do (fisheye lenses, stereo pairs, and the like), which means co-designing its visual adaptation with the sensor hardware — a gap that no amount of capability data can close.

Si₀ capability

Instruction: —

Pointing

Referring

Free space

Waypoints

▶ replay

◀▶

We drive IRON's manipulation with Fe₀ and record the robot's own visual observations — showcasing Si₀'s task-relevant capabilities on real-robot sensors.

Our capability data

Most standard multimodal models are trained to talk about images. An embodied physical model has a harder job: it must understand a scene precisely enough to act in it — to know exactly where an object is, how far, in which direction, how it moves, and what to do with it. We call this bundle of skills spatial intelligence, and we curate Si₀'s training data as a grid over two orthogonal axes: the data source a sample is drawn from, and the atomic capability it teaches. Two of those sources in particular — real-world egocentric-exocentric data and robot-domain data — widen Si₀'s visual domain toward what an embodied model actually sees: first- and third-person human activity and the robot's own viewpoint, rather than curated web images.

Data domain mapping for ablation. To keep our systematic study (§2) rigorously controlled, we treat the general-purpose MLLM data as the baseline backbone (G). The newly introduced spatial data is then categorized into two treated evaluation domains: Ego-exo (E) and Robot (R). Any blended training combination is denoted as Domain·Capability (e.g., E·F for Ego-exo Fine-grained perception, or R·P for Robot Embodied Reasoning/Planning).

Where Si₀ stands

Before asking which skills translate into action, we first establish that Si₀ is a strong spatial-intelligence model in its own right. We benchmark it broadly — spanning general visual understanding, spatial understanding, and embodied reasoning — against spatial-enhanced models at the same scale, and frontier models.

2B-scaleCosmos-Reason2

56.0

Qwen3VL 2B

59.6

RynnBrain 2B

64.0

Si₀ 2B

67.3

4B-scaleQwen3VL 4B

67.7

RoboBrain2.5

67.8

RynnBrain 4B

69.2

Si₀ 4B

71.7

The question, and a two-sided design. Having established Si₀ as a strong performer on static spatial benchmarks, we ask the critical downstream question: which of these capabilities actually translate into physical control, by how much, and do seemingly minor skills earn their keep within the full mixture? We design a rigorously controlled ablation study, probing each capability from two complementary directions:

• Additive — starting from the general MLLM control baseline (G), we add one capability at a time to isolate what each skill contributes in isolation.
• Leave-one-out — starting from the full multi-capability mix, we remove one capability at a time to measure its indispensable value in combination.

Setup. Each capability's training set mixes its single-capability data with a fixed, equal proportion of general data, and we re-pretrain under an identical training budget and hyperparameters — so any difference traces to the capability, not the data volume or training effort. Each 2B backbone is then fine-tuned into a downstream VLA on Robocasa GR-1 at the 100-shot scale and evaluated in simulation across the 24 tasks (50 episodes each), reported as the average success rate.

Findings

Spatial intelligence is synergetic, not monolithic. While grounding leads the charts, no single skill carries the day alone. Our leave-one-out data shows that dropping spatial relations (S) or motion (M) consistently causes the model to give up ground. These dimensions are complementary; the visual backbone requires a holistic understanding of 3D layout and temporal tracking to generalize across multi-step embodied tasks.

Scaling general MLLM data vanilla-dilutes physical density. In stark contrast to spatial tuning, simply doubling down on the general multimodal SFT data (G) under the same training budget yields zero downstream improvement, or even underperforms. This stems from two clean reasons: First, frontier baselines are already heavily saturated via high-quality general multimodal alignment; adding redundant data brings diminishing returns. Second, injecting generic VQA text at this stage vanilla-dilutes the dense, geometry-rich spatial signals, forcing the backbone’s feature space to over-index on abstract semantic concepts rather than the precise structural and spatial layout properties required for physical control.

Real-world exocentric and human-egocentric data earns its place, too — none of it collected from a robot. Every ego-exo capability we add improves downstream control — and ego-exo grounding (E·F) is in fact the single most useful addition of all, on par with its robot-domain counterpart. The foundation can be built largely from abundant human, real-world data, not only from costly robot collection.

Putting it together

2B · Freeze VLM Qwen3-VL

24.5

Si₀

28.3

4B · Freeze VLM Qwen3-VL

22.9

Si₀

29.8

2B · Freeze ViT Qwen3-VL

37.8

Si₀

44.7

4B · Freeze ViT Qwen3-VL

42.1

Si₀

50.2

The gains hold out-of-distribution. The capabilities acquired through spatial-intelligence pre-training transfer to downstream control and remain effective under distribution shift: Si₀ outperforms the strong general-purpose backbone both on the in-domain task families and on out-of-distribution evaluation — held-out container combinations, novel object appearances, and unseen object types.

These viewpoints are unlike the images a general backbone is trained on, and the mismatch runs along three axes at once: optics (strong fisheye distortion vs near-rectilinear web images), viewpoint (head-mounted egocentric vs third-person framing), and geometry (a stereo left/right pair vs a single uncalibrated view). The result is a strong visual-domain shift: these views fall in a corner of visual space that even broad pretraining covers only sparsely, and a general backbone's features degrade accordingly.

Adapting the model, and its effect

We close it by co-designing the model's vision with the sensor — for us, the cameras of the IRON-R1.11 humanoid. The adaptation is deliberately lightweight, and we validate its effect at two levels before the real robot.

The first level is perception. We build an in-house, sensor-consistent evaluation set on native sensor views, and test sensor adaptation across a range of capability dimensions — from fine-grained perception to embodied reasoning. On each dimension we compare the baseline backbone against the sensor-adapted model; the adapted model comes out ahead across the board.

Det Detection — localize objects with a class name. Ref Complex referring — ground an object or person by attributes, texture & relations. X-view Cross-view corresp. — match the same instance across different cameras. Aff Placement affordance — judge where an object can be placed. Avg Average (4 tasks) — mean over the four tasks.

On the real robot

Our final model combines the full spatial-data mix — fine-grained perception, spatial relations, motion perception, and embodied reasoning — with a lightweight sensor adaptation trained on only a small fraction of data (under 5%). Beyond validating Si₀'s spatial-related capabilities through visual language questioning on the robot's sensor-consistent benchmark, we go a step further and validate at the action level: on the IRON-R1.11 humanoid, the same baseline-versus-adapted comparison runs in open loop — open-loop MSE across five action dimensions — and sensor adaptation lowers the error.

L1 Visual Robustness L2 Semantic Grounding L3 Relational Reasoning L4 Compositional Planning L5 Action Generalization Avg mean over L1–L5

What makes a good spatial-intelligence backbone for physical action? Our results point to a two-axis recipe: train the capabilities that downstream policies can actually use, and adapt the visual stack to the robot's own sensors. Neither axis substitutes for the other.

Spatial capabilities transfer to action, but selectively. Fine-grained perception is the most action-sensitive capability: it is the strongest to add and the costliest to remove. Embodied reasoning follows closely, while spatial relations and motion matter most as complementary skills inside the full mixture. In contrast, simply adding more general MLLM data does not produce the same physical-control gains.

Useful embodied priors can come from weakly action-aligned spatial data. Here, real-world ego-exo data is not used to pretrain an action policy or imitate human actions directly. Instead, it supervises the backbone's spatial-understanding capabilities without requiring precise robot action labels. These capabilities still transfer to downstream control, suggesting that abundant human-centered data can build the spatial foundation before expensive robot trajectories enter the loop.

The robot's sensors define a separate distribution problem. Fisheye optics, egocentric viewpoint, and binocular geometry put the humanoid in a visual regime that general backbones do not naturally cover. Lightweight sensor adaptation improves sensor-native perception and lowers action-level error, making perception hardware and backbone training a co-design problem.

Looking ahead. As embodied tasks broaden, the most valuable backbone capabilities may shift with the action distribution. The open problem is to map which spatial skills matter for which task regimes.

Ranked within each scale block: 1st = bold on highlight, 2nd = underlined. The green box marks our model (Si₀). * = cited (Cosmos-Reason2 2B figures are from the Cosmos3 report; other starred figures from each model's own report); — = not reported / reproduced. ^† RefCOCO = average of RefCOCO, RefCOCOg, RefCOCO+; RoboRefIt = test set B. The comparison is directional: task prompts and sampling are aligned to each model's reported setup where provided.