SynAgent: Bridging Individual Proficiency to Multi-Humanoid Collaborative Synergy
SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy
SynAgent is a unified framework for generalizable cooperative humanoid manipulation that transfers skills from single-agent interactions to multi-agent scenarios. It achieves SOTA performance in both cooperative motion imitation and trajectory-conditioned control, outperforming baselines like CooHOI by significant margins in success rates.
Executive Summary
TL;DR: SynAgent is a novel framework that solves the "data drought" in multi-agent humanoid manipulation by teaching robots how to cooperate using single-human interaction data. By combining a geometry-aware retargeting method with a solo-to-cooperative distillation pipeline, it enables humanoids with dexterous hands to carry complex objects (like chairs and lamps) together with high physical fidelity.
Academic Positioning: This work represents a significant "0-to-1" leap in physics-based, dual-agent cooperative manipulation. It moves beyond simple "box-carrying" baselines by introducing a scalable data-driven approach that handles diverse object geometries and high-DOF dexterous hand coordination.
The "Data Scarcity" Bottleneck
In the world of Embodied AI, we have plenty of data for single-person movements (AMASS) and simple object interactions (GRAB). However, High-quality Human-Object-Human Interaction (HOHI) data is nearly non-existent. Without this data, reinforcement learning agents struggle to converge because the joint action space of two humanoids with dexterous hands is massive. Prior work like CooHOI cheated by treating objects as simple boxes, which fails the moment the robot encounters a tripod or a floor lamp.
Methodology: The Solo-to-Cooperative Synergy
The core insight of SynAgent is that cooperative manipulation is essentially single-agent control under external force disturbances.
1. Interaction-Preserving Retargeting
To use single-human data (HOI) for two-human tasks (HOHI), the spatial relationship between the "hand" and the "object" must stay perfect. The authors use an Interact Mesh built via Delaunay tetrahedralization. By minimizing Laplacian deformation energy, they ensure that even if the robot's skeleton differs from the human actor, the semantic "contact" remains intact.
2. Multi-Teacher Distillation & MAPPO
The training happens in stages:
- Stage I: Pre-train on single-human data.
- Stage II: Transition to multi-agent settings using MAPPO (Multi-Agent PPO). Agents use a decentralized policy (shared weights) to perceive each other through the object's dynamics.
- Stage III: Distill these imitation "teachers" into a Trajectory-Conditioned CVAE. This allows the user to simply give a 3D path, and the robots will coordinate their actions to follow it.
Experimental Mastery
The results prove that "more data is not always better—smarter data is." By filtering low-quality sequences and initializing from single-agent priors, SynAgent crushed the competition.
- Imitation Success: Ours (45.00%) vs. InterMimic (7.26%).
- Geometric Generalization: While CooHOI had a 0% success rate on non-standard objects like tripods and clothes stands (due to lacking dexterous hands), SynAgent maintained stable control.
Critical Insight & Future Outlook
Takeaway: SynAgent proves that we can bypass the lack of multi-robot datasets by mathematically preserving the "interaction topology" of single-robot data.
Limitations: The framework still struggles with highly dynamic or "adversarial" coordination where agents must move in extreme directions. The 45% success rate, while 6x better than previous SOTA, shows that physics-based HOHI is still an open frontier. Future work will likely integrate Vision-Language Models (VLM) for higher-level task planning to complement this low-level motor synergy.
Conclusion: SynAgent provides the "0-to-1" foundation for multi-humanoid collaboration, moving us closer to a future where robots don't just work near us, but with us.