Can code-only policies make robot decision making stateful for recursive self-improvement?

COAP replaces the model in the control loop with code

Most robot policies keep a model inside the control loop. A vision-language-action model (VLA) learns a mapping from observations to actions that stays implicit in its weights; an Agent Harness such as Agent-as-Policy or Harness VLA queries a vision-language model (VLM) at run time to decide, or to direct a VLA. The paper's alternative is to treat the embodied world as an Embodied Turing Machine: the tape is the robot and environment state, and the rules are the policy. If that state can be represented accurately, the argument goes, decision making can be written entirely in code.

Code-Only-as-Policy (COAP) is that implementation. Code measures and tracks the robot, environment and task state from camera images and proprioception, and computes every action from it. The same program applies across episodes, and different tasks share one library. At execution the code is frozen: no VLM and no VLA sits in the loop. The paper positions this as a third way of mapping state to action, alongside VLAs (implicit mapping in weights) and Agent Harnesses (partly explicit, but with black-box VLM and VLA still deriving actions).

The state has three parts, each updated from a different source

At each step the benchmark supplies an observation: an instruction, three RGB images (head and two wrists), joint angles, end-effector poses and a gripper command echo. From this the code maintains an explicit state with three parts, each with the same form in every task. The environment state records each object's pose, size and category plus spatial relations such as one object resting on another, and each record also stores the error, source and time of its measurement. The robot state records joint angles, end-effector pose and gripper opening per arm, plus the poses each arm can reach. Together these two form the tape. The task state records the current stage, which arm is assigned to which object, retry counts and information the task must retain, such as the colour under each cup — the analogue of a Turing machine's internal state.

The sources differ. Robot state comes from proprioception every step. Environment state is measured by the perception layer, which is ordinary NumPy and OpenCV code: the robot mesh is projected into the image using measured joint angles and masked out, pixels differing from the table colour become object candidates, and a ray-plane intersection places each candidate on the table before a known 3D shape is fitted by searching position and yaw. Task code updates task state during execution. Because camera models, robot meshes, object shapes and table height are published with the benchmark and do not change between episodes, they are constants in the rules; what varies between episodes is state, measured fresh each time.

One library lets new tasks extend without rewriting old ones

Two design choices make the library reusable. Every task operates on state of the same form, so a module written for one task can serve others. And the code is layered with dependencies pointing downward: task programs call object-specific code, which calls manipulation code such as grasp, pick, place, push and pour, which in turn calls perception and motion code. Task code specifies what to do — subgoal order, task-specific decisions, recovery conditions — while lower layers determine how. A single transfer call expands into grasp-candidate generation, inverse-kinematics and collision filtering, pre-grasp re-measurement, approach, closure, a short lift with a hold check, carry, placement, release and final verification. The paper reports that 83% of lines are shared across tasks.

A new task extends the library in one of three ways: it composes existing modules; it inherits an existing task and overrides a single decision, as a sorting task that reuses an entire flow and replaces only the rule assigning objects to baskets; or it adds a default-off parameter to a shared function, such as a lift direction that lets one task lift diagonally near a basket wall while the default preserves the vertical lift. The default-off mechanism is the compatibility guarantee: for any task that does not pass the new argument, the execution trace must be unchanged on the fixtures exercised by unit tests. The paper's layer table shows the split — perception (5.3k lines) and motion (3.3k lines) are used by 100% of the 40 task programs, manipulation (5.4k lines) by 78%, while object families (28.5k lines) reach 18% and task programs (23.5k lines) only 2.5%.

Coding agents improve the code offline, not at run time

Improvement happens before execution. Coding agents — the paper states Claude Opus 5.5 agents wrote all code in the library and every task, ran tests and evaluations, and debugged their own code — work in rounds. Each agent inspects the recorded state of failed development runs to locate the variable or rule at fault and proposes a diff; the resulting branches are complete, runnable programs that can be evaluated concurrently under identical conditions. Before simulation each branch passes two CPU checks: task code is executed against a numeric scene to confirm every stage reaches its target state, and robot and object meshes are checked for inverse-kinematics solutions and collisions at every waypoint.

Debugging and selection use separate data. Agents debug on a development set they define themselves, watching videos and comparing measured state against simulator ground truth; selection compares success counts on the validation split. A branch is accepted only if its success count on the target task's validation layouts exceeds the current code's count by a margin above rerun noise; otherwise the current code stays deployed and the diff is not merged. The paper argues this makes each accepted step non-decreasing and lets improvements accumulate, and that because the policy is explicit code, the behaviour tested offline is the behaviour at run time — a contrast it draws with a prior robotics RSI loop in which a VLM still makes execution decisions, so a change passing a gate only steers a black box.

70.24% is a library-level number, not a per-task average

On RoboDojo's 42 bimanual tasks, the resulting library reaches 70.24% success with no model at test time. That figure is the overall success after the library was frozen and task updates were merged one at a time, each touching shared code only through default-off parameters; overall success rose at every merge, from 66.5% to 70.24%. It is not the same as the per-task figures reported elsewhere in the paper: on the 40 tasks that have a program, standard success is 76.7% on validation and 74.9% on test, and randomized success is 55.2% and 52.8% respectively. Readers comparing COAP to a VLA baseline should keep these denominators apart — 42 tasks versus 40 tasks with programs, standard versus randomized layouts, validation versus test.

The paper also reports that old tasks did not regress. Between the frozen library and the final code, tasks whose own code did not change kept their success at 78.4% before and 79.9% after on the same layouts (Cochran–Mantel–Haenszel test, p = 0.16; no task changed significantly). Including tasks whose code was later cleaned up, success went from 80.0% to 81.4%. The paper is explicit that this test detects a pooled shift of about 4 points, so it rules out a large regression rather than every change. For four tasks that had no program when the library was frozen, the paper plots mean best validation success over time and gives per-task timelines: first successes arrived between 4.0 and 6.4 hours from the start, and one task, store_tools_in_toolbox, never got a successful program and counts as zero.

Most failures trace to state measurement, not to code logic

The paper classified each failed episode by the smallest change that would have prevented it, reading traces from the start and taking the first unrecovered error; the labels are judgements checked against official videos. Two groups account for the failures. Limits of state measurement make up 62%: perception errors at 35%, where RGB-only input lets a wrong value enter the state (a dartboard in a randomized scene recognized as a headset), and contact errors at 27%, where the representation is too coarse to tell how an object is held (jaws closing on a shirt hem while the soft cloth slips out). Limits of code logic make up 37%: incomplete logic at 29%, such as a push that stops after one try although its own check finds a cube still turned, and poor code design at 8%, such as a fixed size threshold that filters out a small watch.

This split is the paper's own account of its upper bound: how accurately and comprehensively the program represents state, and how robust the code logic is. The authors note that contact with deformable objects is inherently hard to represent but still counts as a limit of state representation, and that perception errors may be more severe in real scenes. The fixes they name are consistent with the framing — better estimation and tools for perception, re-grasping for contact, pushing until the check passes for incomplete logic, and deriving values from the scene instead of hard-coding them.

The evidence stops at simulation, and development is not free

All results are in simulation. COAP's perception relies on the public configuration and assets of the simulated benchmark — camera parameters, robot meshes, object sizes — which the paper calls a major limitation; on a real robot these would have to be measured, and contact-related parameters such as grip widths and carry speeds tuned again. The paper also flags that code may have limited generalization capability.

A second cost is time. Online execution is fast and cheap, but offline development requires evaluation and revision for each iteration, and unlike a model that can be used directly as a policy, COAP has a development cost for every new task. The paper's own timelines make this concrete: new tasks took roughly 2.7 to 7.9 hours to reach a first merge, and one of the four never succeeded. The authors also propose COAP as a data engine for VLAs and Agent Harnesses because the same code applies across episodes, but that is a proposal, not a measured result in the packet.

The decisive variable is state accuracy, not model size

The paper's most useful comparison is internal: 62% of failures come from how well the state is measured and represented, and 37% from how robust the code logic is. That ordering tells readers where to look when judging whether COAP-style policies can carry recursive self-improvement. A coding agent can fix a hard-coded threshold or a missing retry by editing explicit code, and the paper's merge record shows overall success rising from 66.5% to 70.24% under that regime. But a perception error from RGB-only input, or a contact the state cannot represent, is not repaired by better code logic alone — it needs better measurement, which the paper lists as future work rather than a demonstrated gain.

So the practical threshold is this: COAP's ceiling moves when the state representation improves, and its floor moves when the code logic becomes more robust. The 70.24% figure sits inside a simulation with benchmark-provided geometry, on 42 bimanual tasks, with a development cost paid per task. Anyone weighing COAP against a VLA or an Agent Harness should compare on the same denominator — the 40 tasks with programs, standard versus randomized layouts — and should treat the state-measurement share of failures as the number that decides whether the approach transfers beyond the benchmark.

Sources used in this answer