ML Engineer and tech writer Donato Riccio wrote an article entitled "The End of RAG?" discussing what might replace Retrieval Augmented Generation in the near future. The article was received as highly controversial within the AI echo chamber, so I brought Donato on the podcast to discuss RAG, why people are so obsessed with vector databases, and the upcoming research in AI that might replace it.
Takeaways:
- RAG is necessary due to LLMs' limited context window and scalability issues, and the need to avoid hallucinations and outdated information.
- Larger/infinite context window models and linear-scaling models (e.g. RWKV, Eagle) may allow for learning through forward propagation, allowing for more efficient and effective knowledge acquisition
- Agentic flows are likely far more powerful than RAG - and when they actually start working consistently, we may see the need for vector databases dramatically reduced.
- RAG libraries and abstracts can be helpful for getting off the ground but don't solve the hard problems in specific vertical LLM use cases.
- RAG vs Agents, and the complex ways that vertical AI approach RAG in practice
Share your thoughts with us at hello@slingshot.xyz or tweet us @slingshot_ai.
