Sep 07, 2026 07:00:00

GitHub has published a case study showing that shortening the output of AI agents to reduce costs can actually increase costs. How can we effectively improve cost efficiency?

When using an AI coding agent, efficiency is just as important as output quality, and completing tasks quickly, efficiently, and in the right context leads to increased efficiency. In other words, the goal should be to advance tasks by leveraging the right amount of context, rather than reducing the number of tokens in each interaction. GitHub has published a blog post about the improvements they've made to Copilot Copilot based on 'optimization focused on results, not tool calls.'

How we make AI coding more cost efficient without sacrificing task quality - The GitHub Blog

https://github.blog/ai-and-ml/github-copilot/how-we-make-ai-coding-more-cost-efficient-without-sacrificing-task-quality/

The following four changes were made to the GitHub Copilot CLI to maximize AI coding efficiency:

- Reduce repetitive output and maintain useful context.
- Remove unnecessary formatting that does not add value to the task, while retaining the information.
- Compress the prompt without changing the necessary behavior.
- Provide the results of completed background work without requiring additional turns to be taken.

The proposed changes were evaluated offline using benchmarks for agent-based coding, and the most promising changes were then validated through controlled online experiments before release. The following figure shows the results of independent A/B tests measuring the impact on the AI credit metric for four cost optimizations performed with the GitHub Copilot CLI. Cost savings were observed as follows: 'Removing prefixes such as line numbers in view tools' resulted in a 3.1% reduction, 'Selective compression of output' in output resulted in a 5.5% reduction, 'Compression of prompts in task tools' in output increments of 2.9%, and 'Reducing extra round-trip processing during notifications' in output increments of 2.3%.

◆ Reduce repetitive output and maintain useful context
A common approach to reducing the cost of AI agents is to shorten the output from each tool call. We evaluated the effectiveness of using

RTK (Rust Token Killer), a utility that shortens shell output before the agent reads it, against GitHub Copilot using an agent coding benchmark. While RTK did shorten some responses in the harness and benchmark configurations, there were cases where the model reopened the output or re-executed commands to recover what was needed when the omitted text was important. The recovery step added more processing steps and carried over more context. Although the responses of individual tools were shortened, more tokens were consumed and the time taken was longer. As a result, while tokens were saved in key areas, overall more tokens were consumed.

This result means the token target per tool call is incorrect. Efficiency evaluation needs to be performed across the entire task, from the user request to the final result. Further benchmark analysis revealed that the output of installation, build, test, and lint often contains repetitive noise, while the output of source code and the results of arbitrary command execution are more likely to contain the information the agent needs. The following three policies were established through trial and error:

- The output of source code and the results of arbitrary command executions are retained.
- Reorganize search results without deleting content.
- Selectively compress recurring noise.

This 'compressor' has been validated by tracking and investigating the frequency of whether the agent 'opened the saved original,' 're-executed commands,' 'repeated searches,' 'narrowed searches,' and 'performed additional turns.'

◆Remove unnecessary formatting that does not add value to the task while retaining the information.
Traditionally, view tools numbered the beginning of each line before displaying its contents in the model. Older file editing tools used line numbers to identify changes, but current tools no longer use them because they instead match changes against surrounding code. Therefore, line numbers were no longer used in typical workflows, but they remained. While the impact of line numbers at the line level was minimal, their repetition across all lines in a file led to an accumulation of unused formatting during sessions, so line numbers were removed.

Removing row numbers reduced model inference cost by approximately 5%, without affecting the success rate. As a result, developers were able to allocate more window space to the context needed for actual work.

◆Compress the prompt without changing the necessary behavior.
Prompts instruct the agent on how to act and are sent to the model at each turn. Shortening prompts improves efficiency only if the agent maintains the behavior the developer depends on. The GitHub Copilot CLI's task tool launches a special agent for parallel work, but agent guidance was stored in the tool's description, schema, agent definition, system instructions, and companion tool.

By using a 'meta-prompt loop' in which Copilot repeatedly creates and improves its own prompts, the number of prompts was reduced by approximately half. Initially, this resulted in a regression where parallel execution guidance was rewritten to a sequential execution policy. However, this was ultimately resolved by issuing a concise instruction stating, 'Independent agents can run in parallel, so please consider the side effects.' As a result, approximately 1300 tokens worth of prompts were reduced per turn, which corresponds to a reduction of approximately 1.8% in the total number of prompt tokens per session and a 2.9% decrease in the normalization cost per active time. Furthermore, no degradation in quality was detected in the measured evaluation.

◆ Provides completed background work results without requiring additional turn acquisition.
Agents often perform independent tasks in the background, such as long-running shell commands. Previously, because the completion notification for a task did not include the result, the agent had to spend an extra turn to retrieve the result of the notification Copilot received. In other words, a completed task required two model calls: one to 'request the result' and another to 'process the result.'

The improved Copilot now requires only one model call because the harness batches both completions and delivers the results together. By having the harness deliver the completed results directly without compression, summarization, or deferral, we have been able to reduce the average token usage, measured in AI credits, by approximately 2.3%.

◆Summary
The key is to optimize the entire task. Making a change to save tokens in one workflow may increase the cost in another workflow. This means that you need to measure the changes in cost efficiency within the workflows that are actually running. With that in mind, here are five things to keep in mind when building an efficient AI coding agent:

Optimize the task instead of calling a tool.
- Optimize not only the model output but also the orchestration.
- Compression is performed based on what the output represents.
Rewriting the prompt may lead to unexpected results.
• The evidence is specific to the workload.