OpenRAG: An open-source stack for RAG — Phil Nash

By ai.engineer

Categories: AI, Tools

Summary

RAG isn't dead—it's just hard. OpenRAG combines Dockling, OpenSearch, and Langflow into a production-ready stack that handles the real complexities: PDF parsing, chunking strategies, embedding updates, and query rewriting that most teams underestimate.

Key Takeaways

  1. RAG complexity varies dramatically by project; PDFs, chunking strategies, and embedding model updates create persistent technical debt that treating RAG as 'solved' ignores.
  2. Dockling's hierarchical document parsing produces intermediate XML-like representations (dock tags) that preserve document structure, enabling smarter chunking strategies instead of naive text splitting.
  3. PDF processing requires multiple specialized pipelines: standard layout+OCR models for scanned docs, or vision language models (Granite Dockling 258M) for unified extraction—choose based on document type.
  4. OpenRAG supports flexible embedding providers (OpenAI, local models) and runs entirely offline with air-gap capability, avoiding vendor lock-in and reducing per-query token costs for document-heavy systems.
  5. OpenSearch combines vector search and keyword search with configurable filtering/aggregation, providing the foundation for hybrid search approaches that improve RAG retrieval quality beyond pure semantic similarity.

Topics

Transcript Excerpt

Hi there. My name is Vanash and I'm a developer relations engineer at IBM. I've been working on uh tools around uh AI and Rag for the last couple of years and I've got something uh I'd like to show to you today. Uh now, first things first, I've heard that Rag is dead many a time and I'm sure you have too. Context windows are huge these days, so you might as well just dump all of your information into there. I don't take this kind of thing very seriously. Uh if every business has less than a mill...