Interactive Code Execution for Reducing Model Calls in Robot Agents

This introduction relies on the available preprint of the paper, arXiv:2610.01939v1, posted on October 1, 2026; the paper does not mention peer review, and its independent peer-review status is not verified here. The paper introduces PyRUA-Lean, an interactive Python-execution interface for vision-language-model robot agents. Its central problem is token overhead from repeated model calls, automatic camera-image returns, and the accumulation of intermediate results in the conversation.

In the tool-calling baseline (RPent), an operation dependent on a previous result generally requires a new model turn. In PyRUA-Lean, rather than selecting one tool per turn, the model writes a Python cell that composes several motion, perception, or vision-language-action primitives. The program can inspect motion outcomes, execute a next step only after success, perform local retries, and retain intermediate data in a persistent namespace within the same episode.

Only explicitly printed outputs and images requested through robo.show are returned to the model. Thus, data available in the runtime need not enter the model’s next context. This mechanism does not delete earlier conversation history: Codex CLI still retains the full conversation and compacts it only near 272,000 tokens. The paper presents an example where geometric placement computation, lowering, release, and a conditional image request run in one cell.

The main evaluation covers 700 paired instances and 1,400 simulated episodes: 40 LIBERO-PRO tasks, 50 RoboTwin 2.0 tasks, and 50 RoboCasa365 Target50 tasks, each at five seeds. Both agents used the GPT-6 Astra planner at high reasoning effort, identical RPent primitives, frozen VLA policies, and no cross-episode memory. The model-call budget was 40, except for RoboCasa365 composite tasks with 100 calls; the time limit was two hours.

Overall success rose from 63.1% for tool calling to 71.7% for PyRUA-Lean: an 8.6-percentage-point, or roughly 14% relative, increase. On instances solved by both agents, mean calls declined from 17.0 to 8.7 and input tokens from 788,000 to 276,000, representing 49% and 65% reductions. Estimated list-price cost fell from $1.63 to $0.74 per episode. The paper attributes most token savings to fewer model calls, rather than necessarily smaller prompts per call.

Important limitations remain. Evaluation used only GPT-6 Astra, RPent stacks and primitives, and simulation, with one run per agent-instance pair. Physical-robot performance, other planners, alternative primitive configurations, and variation across repeated runs were not tested. Moreover, the comparison evaluates complete interfaces and does not fully isolate the causal effects of primitive composition, persistent state, and selective feedback. Gains are smaller on VLA-dominated atomic tasks, and the main efficiency metrics are reported only on instances both agents solved.

Original arXiv paper