Aleksei Naumov

A Novel RAG Approach That Understands The Whole Document Context RAG has rapidly evolved to be the standard way to apply LLMs in production. However, most methods are still limited because most existing methods retrieve only short contiguous chunks from a retrieval corpus, limiting holistic understanding of the overall document context. A new approach named RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval (link in alt) proposes recursively embedding, clustering, and summarizing chunks of text, constructing a tree with differing levels of summarization from the bottom up. At inference time, the RAPTOR model retrieves from this tree, integrating information across lengthy documents at different levels of abstraction. Controlled experiments show that retrieval with recursive summaries offers significant improvements over traditional retrieval-augmented LMs on several tasks. On question-answering tasks that involve complex, multi-step reasoning, we show state-of-the-art results; for example, by coupling RAPTOR retrieval with the use of GPT-4, we can improve the best performance on the QuALITY benchmark by 20% in absolute accuracy. We will continue to see more methods like this one that are designed to improve document understanding and RAG. The good thing about this method is that it focuses on the retriever piece, while most of the other methods tinker with the context by adding noise or web results.