Why More Context Makes Your Agent Dumber and What to Do About It — Nupur Sharma, Qodo

By ai.engineer

Categories: AI, Tools

Summary

LLM agents suffer from a 'U-curve' attention problem where they focus on initial and final context while ignoring middle inputs—dumping more data actually makes performance worse. Rather than building expensive context engines, use hierarchical summarization or knowledge graphs to strategically optimize what matters.

Key Takeaways

1. LLMs exhibit a U-curve attention pattern: they retain initial prompts and final inputs but actively ignore middle-context, making larger context windows counterproductive without optimization.
2. Context engines solve ranking and search patterns but hit scaling challenges at 600+ repositories, making them impractical for most teams not purely building context infrastructure.
3. Hierarchical summarization creates file/folder-level summaries so agents scan metadata first, reducing wasted processing—requires upfront LLM costs per file change but cuts redundant analysis.
4. Knowledge graphs work best for logical dependencies across multiple files/repos but demand high initial developer input—only justifiable for complex codebases with clear architectural relationships.
5. Multi-agent architectures fail when agents clash on understanding—assign each agent a single task (security, review, coding) rather than dumping all context to one system.

Related topics

• Tools
• AI
• LLM Context Window Optimization
• Agentic Workflow Architecture
• Multi-Agent Coordination Patterns
• Knowledge Graph for Code Understanding
• Hierarchical Summarization Strategy

Transcript Excerpt

I'm Nupur. I work with Kodo. Uh at Kodo we do agentic reviews. Uh I have a background in dev sec ops. So I'm coming from an industry where everything was deterministic. The pipelines they run they crash. If they crash, we fix them. Uh to a place where we are doing agents where nothing is deterministic. So in my last few years I have learned where and how agents fail, what are the learnings and today I'll be sharing some of my learnings with you. So um if you see the evolution of agents, it started with static prompts where it was a 4K context window and we tried to put whatever was important or whatever we deemed important and the AI models will process it and provide you with a result. Right? When we started with that that means that it was on us to tell LLMs what do they should they shou…

More from ai.engineer

• Why Eval++ Is the Next Great Compute Primitive — Sunil Pai & Matt Carey, Cloudflare
• How to Keep Shipping When You Walk Away from Your Desk — Zack Proser, WorkOS
• RAG is dead, right?? — Kuba Rogut, Turbopuffer
• Text Diffusion — Brendon Dillon, Google DeepMind
• The maturity phases of running evals — Phil Hetzel, Braintrust