[CoRL 2025] Act–Observe–Rewrite: The Era of the Self-Debugging Robot Agent

TL;DR

Imagine a robot that doesn't just "fail" but looks at a video of its failure, reads its own source code, identifies a sign error in a coordinate transformation, and rewrites its own controller to fix it. That is the core of Act–Observe–Rewrite (AOR). Using multimodal LLMs (like Claude Sonnet 4.x), this framework treats the entire motor-control implementation as an editable script, achieving SOTA-level manipulation success without a single gradient update or human demonstration.

Background: Moving Beyond "What" to "How"

In the current robotics landscape, we usually see two paths:

1. The VLA Path: Training massive models (RT-2, Octo) on millions of trajectories. Great for generalization, terrible for debugging specific failures.
2. The Code-as-Policies Path: Asking an LLM to write a plan once. If the plan fails due to physical noise or a geometry bug, the robot is stuck.

AOR occupies a unique "System 2" reasoning space. It asks: Why did we fail, and how can we change the underlying math to succeed?

The AOR Framework: Two Timescales, One Goal

AOR splits the robot's existence into two loops:

• The Fast Loop (Real-time): A Python controller executes at high frequency, using classic techniques like PID, state machines, and HSV segmentation.
• The Slow Loop (Reasoning): Between trials, the agent acts as a "Senior Tech Lead." It reviews RGB-D keyframes and logs to find systematic biases.

Figure 1: The AOR loop. The LLM acts as the programmer, rewriting the green "Policy" box based on visual evidence from the environment.

Methodology Insight: Why Code Matters

The breakthrough here is the unit of reasoning. Unlike prior work that merely selects a "skill" (e.g., "Pick up bottle"), AOR can modify the back-projection formula itself.

In the "Stack" task, the agent encountered a 5–8 cm systematic error. A neural network would require thousands of examples to compensate for this bias. The AOR agent, however, realized that the simulator used OpenGL conventions (where the Y-axis is flipped). It literally rewrote the line: y_p = (v - c_y) * d / f_y to the correct OpenGL-adjusted version.

Experiments: Results that Speak

The researchers tested AOR on three progressively harder tasks in robosuite:

Figure 2: Iterative improvement on the Stack task. Notice the jump in success (v7-v10) after the agent fixed the coordinate projection code.

Critical Analysis: The Limits of Reflection

Despite the 91% success on stacking, the agent hit a "local optimum." In the final 9% of failures, the gripper fingers would bump the bottom cube. While the agent identified the contact as the cause, it couldn't "think outside the box" enough to invent a new placement trajectory (like a lateral nudge) before the iteration budget ended.

Limitations:

• Search Incompleteness: The agent can get stuck in a "coding loop" without exploring diverse physical strategies.
• Simulation vs. Reality: Real-world lighting and actuation noise will provide a much harsher test for the HSV-style segmentation the agent prefers.

Future Outlook

AOR proves that for physical AI, interpretability is a feature, not a byproduct. By using code as the policy, we gain a robot that can participate in its own debugging. Future versions could combine this with "VLA priors"—using a foundation model for the first attempt and AOR for the "last mile" of precision and adaptation.

Final Takeaway

If you want a robot to learn fast, don't just give it data; give it a debugger and the power to rewrite its own reality.