Recently I read Databricks’ article, Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebase. The article is long, but one point caught my attention: the cheapest AI model is not always the cheapest coding agent.

Most people compare AI coding tools by model name, subscription price, or token price. But Databricks used a more practical metric: cost per completed task. I think this is much more useful, because coding agents do not only consume tokens. They also need context, tool calls, retries, and time.

Cost per token can be misleading

A model can be cheaper per token but still more expensive in real usage. If it needs more context, more turns, or more failed attempts, the final task cost can be higher.

Databricks gave an interesting example. Sonnet 5 was cheaper per token than Opus 4.8, but in their benchmark, Sonnet 5 cost $2.09 per task, while Opus 4.8 cost $1.94 per task. So the cheaper model was actually more expensive per finished task.

That is the key point: for coding agents, token price is not the full cost.

GLM 5.2 was surprisingly strong

Another interesting result was GLM 5.2. According to Databricks, GLM 5.2 reached the top capability tier and was statistically tied with Opus 4.8 on quality, but the cost was lower.

GLM 5.2 cost $1.28 per task, compared with $1.94 per task for Opus 4.8. Of course, this does not mean GLM 5.2 is always the best choice. This was Databricks’ benchmark, based on Databricks’ own codebase and tasks.

But it does show something important: open models are becoming serious options for coding agents, not just cheap alternatives.

The coding tool also matters

Another important point is that an AI coding agent is not only the model. The tool around the model also matters. Databricks calls this the harness.

The harness decides things like which files to read, how much context to send, how to call tools, and how to run tests. Databricks found that the same model can have very different cost per task depending on the harness.

So when we compare AI coding agents, we should not only ask which model is smarter. We should also ask which tool uses the model more efficiently.

My takeaway

For me, the biggest lesson is simple: do not compare AI coding agents only by token price.

A cheaper model can become expensive if it needs more attempts. An expensive model can be better value if it solves the task faster. And the tool around the model can change the final cost dramatically.

So the real question is not “Which AI coding agent is the best?” The better question is: Which AI coding agent can complete my real coding tasks at the lowest reliable cost?