Unified Mastery: Bridging the Gap Between Acting and Planning via Operational Models

TL;DR

Researchers have long struggled with the "dual-model" problem in AI: using abstract PDDL for planning while using concrete code for acting. This paper presents a unified system—RAE + UPOM—that plans directly using the actor's execution code. By leveraging Monte Carlo Tree Search (UCT) and Deep Learning, the system achieves unprecedented efficiency and robustness in unpredictable, real-world-like robotics domains.

The "Model Mismatch" Problem

In classical AI, there is a fundamental divorce between "thinking" and "doing."

• Descriptive Models (Planning): Focus on what happens (Preconditions $\to$ Effects). Great for search, terrible for handling complex "while-loops" or real-time reactive logic.
• Operational Models (Acting): Focus on how to do it (Procedures/Rules). Great for handling sensors and errors, but hard to "look ahead" because they aren't designed for state-space search.

The authors argue that this gap is the primary reason why autonomous systems fail in safety-critical applications like self-driving cars or search-and-rescue. Their solution? Make the planner speak the actor's language.

Methodology: The RAE and UPOM Synergy

1. Hierarchical Operational Models

Instead of simple STRIPS operators, the system uses Refinement Methods. These are essentially Python-like procedures that can include subtasks, loops, and conditional logic.

2. The Refinement Acting Engine (RAE)

RAE is the "driving" force. It maintains an Agenda of tasks and uses a LIFO stack to manage hierarchical refinement. When it hits a decision point (multiple ways to solve a task), it asks the planner for advice.

3. UPOM: Planning by Simulation

UPOM (UCT Procedure for Operational Models) is the "brain." Since the models are procedures, UPOM cannot use standard A* or BFS. Instead, it uses Monte Carlo Tree Search (MCTS). It runs thousands of "rollouts"—simulated executions of the actual acting code—to see which path yields the highest utility (Efficiency or Success Ratio).

Figure 1: The integration of refinement acting, planning, and learning.

Accelerating Intelligence via Learning

MCTS is computationally expensive. To make this work in real-time, the authors introduced three Supervised Learning strategies:

• Learnπ: A policy network that predicts the best method for a context immediately (for when the actor has no time to think).
• LearnH: A heuristic network that estimates the value of a state-task pair to prune the MCTS search tree.
• Learnπi: Learns to instantiate specific parameters for methods (e.g., which robot should perform a task).

Experimental Proof

The authors tested the system in five complex environments including Search & Rescue (S&R) and Warehouse Delivery.

Key Findings:

• Robustness: In domains with "dead-ends," RAE+UPOM significantly outperformed reactive systems. Planning allows the agent to avoid irreversible mistakes.
• Efficiency: The "Retry Ratio"—a measure of how often an agent fails and has to try a different method—dropped sharply as the number of rollouts increased.
• Speed: UPOM proved to be significantly faster and more scalable than previous state-of-the-art acting planners (RAEplan).

Figure 2: Efficiency and Success Ratio relative to the number of rollouts.

Deep Insight: Why This Matters

The core genius of this work lies in its Inductive Bias. By assuming that human experts can provide a "skeleton" of how to solve a task (the methods), the system doesn't need to learn from scratch. It only needs to learn the preferences between those methods. This makes the AI much more reliable and easier to debug than a pure Reinforcement Learning agent, while being more flexible than a hard-coded robot.

Critical Perspective & Future Work

While the system is robust, it relies on having a generative simulator of the environment. If the simulator is wrong, the plan is wrong. Future research must focus on Learning the World Model alongside the refinement methods to bridge the gap between simulation and reality further.

Conclusion

This paper is a milestone in the "Actor's view" of AI. It demonstrates that by unifying acting and planning under hierarchical operational models, we can create agents that are both strategically smart and tactically reactive.

Takeaway: Stop writing planning models and acting code separately. Write the acting code, and let the planner simulate it.