Scaling Slack and GitHub with Maude Lemaire

By Pragmatic Engineer

Categories: Product, Startup

Summary

Scaling systems like Slack and GitHub requires rethinking architecture fundamentally—GitHub's 16 database shards for all pull requests nearly broke under load, and load testing 500K websocket connections cost $150K per test. AI currently fails at performance optimization investigation, making human expertise irreplaceable for infrastructure scaling.

Key Takeaways

1. Load testing Slack's 500K websocket connections using headless Chrome instances cost $150K per test ($0.37 per instance for 5-hour load test)—revealing that brute-force testing approaches become prohibitively expensive at scale.
2. GitHub's monolithic Rails architecture used only 16 database shards for all pull requests and issues globally, creating a massive scatter query bottleneck that prevented horizontal scaling—a constraint that persisted due to architectural debt.
3. Scatter queries in large Rails monoliths create exponential query multiplication—preventing simple sharding solutions and forcing architectural rewrites, a problem AI LLMs cannot currently solve during performance optimization investigation.
4. Platform and product engineers should rotate between teams to build systems thinking—scaling expertise requires understanding both infrastructure constraints and product requirements simultaneously, not isolated specialization.
5. Current LLMs provide minimal value for performance optimization and investigation phases—the core bottleneck in scaling systems. AI excels at code generation but fails at diagnosis of complex infrastructure problems.

Related topics

• Product
• Startup
• Database sharding strategies
• Websocket scalability architecture
• Scatter queries Rails optimization
• Load testing cost optimization
• Monolith to distributed systems migration

Transcript Excerpt

We needed 500,000 active websocket connections. We had a solution that would spin up Slack as a headless Chrome instance in a bunch of Kubernetes pods. It was 37 cents per headless Chrome instance per 5 hour tunch of load testing. >> Yeah, we're talking about $150,000 for that one test. >> There were 16 shards for all of pull requests and issues >> across all of GitHub. across all of GitHub only 16 which is not very many given the amount of data that we know is associated with that whole space. The answer was oh well we can't have more because we already have so many scatter queries. >> What parts of the job does AI not help you with? >> It has not helped me with performance optimizations basically at all. At least the investigation part. The LLMs right now I find are good at >> how do you…

More from Pragmatic Engineer

• The revised rules of engineering management with Will Larson
• Uber’s AI infra, from the inside
• Building resilient systems with Sam Newman
• The state of the tech industry in 2026 - LDX3 New York
• Distributed databases with Peter Mattis