FOLLOW A QUESTION FURTHER

Library

Practical lessons, original research, and the details that make systems work.

Refine your reading (1)
8 articlesReset filters

Articles

Implementation
Cloudflare· Ryan Skidmore

Cutting our inference bill by 68% by being obsessive about optimization (opens in a new tab)

When we previously blogged about our AI code reviewer in April, an average review cost a palatable $1.19. By the end of August, that figure had shot up to about $2.46. We hadn't made any significant architectural changes: the coordinator was still handing work to the same set of specialist reviewers, and the same risk tiers decided how intensive of

Implementation
GitHub Engineering· Erik Kristensen

How we make AI coding more cost efficient without sacrificing task quality (opens in a new tab)

Output quality is important when working with AI coding agents, but true efficiency comes from getting work done quickly, efficiently, and with the right context. That’s why token count of individual interactions alone isn’t a meaningful measure of efficiency. The goal shouldn’t be to use fewer tokens, but to tap into the right amount of context to

Research
Berkeley AI Research

Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction (opens in a new tab)

Overview of ABBEL compared to traditional recursive summarization. Beliefs replace the full interaction history as the agent’s working context, and belief grading improves performance by supervising the contents of each belief state.. As task horizons grow, LLM contexts can’t scale forever. Self-summarization enables concise, interpretable contexts