SpaceDex: Mastering Dexterous Grasping in the "Vertical Frontier" of Tiered Shelves

TL;DR

Most AI-based robotic hands perform beautifully on flat tables but "choke" when placed inside a crowded bookshelf or refrigerator. SpaceDex is a new hierarchical framework that solves this by combining high-level Vision-Language Model (VLM) reasoning with a low-level "Arm-Hand Feature Separation" network. It achieves a 63% success rate in constrained 3D spaces, outperforming state-of-the-art tabletop models by a staggering 24 percentage points.

The "Tabletop Assumption" Problem

Robotics research has a hidden bias: the Tabletop Assumption. Most SOTA models (like RT-2 or OpenVLA) are trained and tested on open surfaces where objects are easily seen from above.

However, real-world utility happens in tiered workspaces—shelves, cabinets, and drawers. These environments introduce:

1. Severe Occlusion: The robot's own arm often blocks the camera view.
2. Narrow Clearances: Moving the arm is a 3D puzzle; any slight deviation leads to a collision with the shelf "ceiling."
3. Tiered Dependencies: Grasping an object at the back of a shelf requires understanding height and depth relations that 2D-centric models ignore.

Methodology: Hierarchical Spatial Intelligence

1. High-Level: The Tier-Aware Planner

SpaceDex doesn't just "look" at the scene. It uses Qwen2.5-VL to query three external cameras in parallel. By analyzing which view has the highest confidence and least occlusion, it selects a primary perspective to generate a persistent segmentation mask. This "semantic grounding" ensures the robot knows exactly what to track even as it reaches deep into a dark shelf.

2. Low-Level: Arm-Hand Feature Separation

This is the "Secret Sauce" of the paper. Typically, a policy uses one latent vector to control the whole robot. But in a shelf, the Arm needs to worry about Global Navigation (don't hit the wall), while the Hand needs to worry about Local Geometry (grasp the curve).

SpaceDex introduces the Arm-Hand Feature Separation Network within a Diffusion Transformer (DiT). It splits the representation into two specialized streams:

• ${\varphi }_{arm}$: Optimized for spatial trajectories.
• ${\varphi }_{hand}$: Optimized for object shape matching and tactile-aware closure.

Fig 1: The Arm-Hand Feature Separation Network architecture within the DiT policy.

Experimental Results: Breaking the 60% Barrier

The researchers tested SpaceDex against DexGraspVLA in a three-layer shelf setup.

Tactile Resilience

In 3D workspaces, "vision-only" control is blind once the hand surrounds the object. SpaceDex utilizes 3D tactile sensors to sense the "staggered rise in force." This allows the hand to stabilize its grip even when the cameras are completely blocked by the robot's own forearm.

Fig 2: Quantitative success rates across various object geometries.

Critical Insight: Why Does Decoupling Matter?

The ablation studies provide a profound lesson: Removing the Feature Separation Network dropped success rates by 19%.

In open spaces, the arm and hand can be controlled as one unit. But in a shelf, the "gradient" of the arm task (avoiding huge static obstacles) often "drowns out" the delicate signals required for a multifingered grasp. By forcing the network to maintain separate internal representations for arm and hand, SpaceDex prevents the macro-movement from sabotaging the micro-manipulation.

Conclusion

SpaceDex proves that dexterity is not just about the fingers—it's about spatial context. By bridging the gap between high-level reasoning (where am I reaching?) and low-level physics (how do I touch?), this work moves us closer to robots that can actually navigate the cluttered, tiered reality of human homes and warehouses.

Future Outlook: The next step will be handling dynamic environments—imagine a robot reaching into a shelf while a human is moving objects around it.