Nvidia 的 SoL-Pi 系统通过优化 harness 将近乎一半的编码 agent token 用量削减

A new Nvidia paper describes a system that automatically optimizes the control layer of coding agents, known as the harness. Token usage drops by almost half while performance stays roughly the same, according to the researchers. The longer AI agents work unsupervised, the more expensive they get. Single predictions turn into long chains of reasoning, tool calls, and feedback loops, and token usage balloons along the way. A new study from Nvidia researchers tackles these costs not at the model level but at the harness , the control layer between the model and its environment used by systems like Codex, Claude Code, or OpenClaw. The harness controls how an agent sees states, runs actions, and processes feedback. Most efficiency methods so far have focused on cutting the cost per token through faster attention kernels and serving infrastructure, model compression like quantization, or swapping in cheaper models. Optimizing the harness is hard in practice because tool usage, context management, verification, and abort logic are all tightly coupled. A change that saves tokens in one place can trigger errors elsewhere or just push costs into a later phase. Typically, humans sift through long execution traces and translate recurring failure patterns into code. The system, called SoL-Pi, automates that work. A research agent watches another agent's traces, proposes changes, and tests them in prepared environments. Capability and efficiency checks determine which candidates survive. The approach draws on recursive self-improvement , according to the authors. Across 535 executable environments, the system explored 152 directions, including 495 tasks derived from GitHub issue-pull-request pairs and 40 synthetic test cases. In total, the process generated more than 3,000 runs and over 60,000 agent-environment interactions. According to the researchers, this scale shows how broadly the system searched, but more search doesn't automatically yield better results. That's a risk here, because earlier work showed that automatically optimized harnesses tend to overfit to their training tasks and offer little benefit on unfamiliar ones. SoL-Pi addresses this by strictly separating search feedback from evaluation. The researchers used EdgeBench as their test benchmark and walled it off from the search process entirely. Of its 51 public tasks, they used 11 for one-time validation of finished candidates. The remaining 40 were reserved for final evaluation, and those results never fed back into the search. The search produced four mechanisms. Action Fusion merges two consecutive steps into one, such as a code edit followed by a test run, which eliminates an entire language model call. Online Context Compact runs after each planning step and trims accumulated context whenever it can do so without losing important information. ObservationPack archives long tool outputs and drops in a short summary on later steps rather than resending the full text each time. The Evidence-Preserving Reducer routes large error and test logs to a cheaper model that boils them down to the key findings, with an automatic verification step catching any critical clues that slip through. On EdgeBench's 51 public tasks, SoL-Pi performs about as well as the original Pi harness, according to the researchers. How much token usage drops depends on the configuration. The most efficient variant combines all four mechanisms, uses 49 percent fewer tokens, and reaches 93.7 percent of Pi's score. Users who prioritize performance and pick only the strongest single mechanism beat Pi's score by 5.3 percent while still saving tokens. Across both variants, token usage drops by 44.7 to 49 percent. In dollar terms, the authors estimate savings of $8.75 to $13.50 per hour compared to native Codex and Claude Code harnesses, and $4.36 to $5.71 per hour compared to Pi, based on current API prices.