How to Get Frontier Coding Results at 70% Lower Cost - Nouha Dziri, Cohere

By ai.engineer

Categories: AI, Tools

Summary

Cohere researcher reveals how hierarchical multi-agent teams solve complex coding tasks at 70% lower cost than scaling individual agents—by dynamically allocating compute across planning, implementation, and verification stages rather than simply adding more workers to independent attempts.

Key Takeaways

  1. Replace parallel sampling with hierarchical task decomposition: instead of running multiple independent agent attempts (which balloons token costs), assign a lead agent to break work into subtasks and delegate to workers with fresh context windows, reducing token overhead by 70%.
  2. Frame multi-agent scaling as a compute allocation problem—optimize for 'reward per added dollar' rather than team size. Determine whether the next inference budget should fund planning, implementation, or verification stages based on empirical efficiency metrics.
  3. Design teams as top-down hierarchies with a lead agent dynamically deciding team size and role allocation per task. This minimal harness approach allows single-variable testing to isolate which design choice (organization, communication pattern, worker model tier) moves reward vs. cost.
  4. Avoid agent-to-agent coordination—have workers report only to the lead. Direct peer communication creates redundant work and stepping on toes; centralized reporting maintains context and prevents duplicate effort on identical subtasks.
  5. Mix model tiers strategically by role: less capable (cheaper) models can handle implementation or verification under lead supervision, while reserving frontier models only for planning or lead roles. This model-selection strategy maximizes cost efficiency without sacrificing task completion.

Related topics

Transcript Excerpt

Hi everyone. My name is Nuha, I'm a senior researcher at Cohere and associate industry member at Mila and an adjunct PI at the Allen Institute to begin. So excited to tell you today about how to optimize your agent team and get frontier coding at 70% lower the cost. This is a work that I did at Cohere. All right, so traditionally there are two ways to make an agent stronger at inference time. First, you can let it think longer, meaning you scale compute sequentially with longer chain of reasoning. And second, you can sample more. You run many independent attempts in parallel and then you aggregate them. But as agents take on longer and longer task, they can navigate large repo, they can edit the code, they [snorts] can debug their own work over hours. So what can happen here is that the co…

More from ai.engineer