Memory and Continual Learning: Engram's Dan Biderman and Jessy Lin
Summary
Engram argues the AI bottleneck isn't raw intelligence but continuous learning on private context—teams will generate tens of millions of tokens daily, making externalized memory expensive and inefficient. Their solution: train per-team adapter models that deeply embed company knowledge into weights rather than relying on context windows.
Key Takeaways
- Current AI solutions use context engineering (huge prompts, multi-turn conversations) but this doesn't scale—users will collectively generate tens of millions tokens per day, making token-based memory prohibitively expensive and confusing for models.
- Memory as externalized databases or context windows is insufficient—true continual learning requires baking knowledge into model weights the same way pre-training does, creating deep contextual understanding like a multi-year employee would have.
- Engram's technical approach uses adapter fine-tuning (LoRAs, prefixes, sparse architectures) combined with training signal extraction (SFT, RL, on-policy distillation) to train per-team models that continuously improve on domain-specific tasks.
- Partner with knowledge-dense platforms (Notion, Microsoft, Harvey) that already contain organizational context and user interactions—this becomes the raw training data for models that understand company processes, initiatives, and workflows.
- The bottleneck for AI usefulness has shifted from raw intelligence to understanding evolving context—new tasks, domain-specific knowledge, and job-specific context are what limit current model utility, not their baseline capabilities.
Related topics
Transcript Excerpt
What about pre-training or even post-training makes it possible for the models to generalize in these magical emergent ways and controlling that process so that a company has a set of private data? How do we make the models [music] learn that just as well as the models know like the capital of France or you know like how to write Python? Um so I think it it's a really fun problem to think about. >> [music] [music] >> Welcome to training data. We are delighted to have Don Beerman and Jesse Lynn, co-founders of Engram today. Engram is a neolab focused on memory and continual learning and two of the hottest topics in all of AI research today. Okay. And Sean and I are delighted to dig in on those topics with you today. >> Awesome. Happy to be here. >> Great. So maybe to kick off, the engram we…