Introduction#
Large language models sometimes generate false or incorrect information when producing text, which is akin to "hallucination." While RAG technology can help models reduce these errors by retrieving relevant information from external databases, it heavily relies on the ability to find the correct information. If the retrieved documents are not accurate enough, it can lead to even more erroneous generated results.
CRAG (Corrective Retrieval-Augmented Generation) was proposed to address this issue. It introduces a new approach by using a "retrieval evaluator" to check whether the retrieved documents are reliable. If the documents are accurate, it extracts important information from them; if not, CRAG can further find better content through web searches. This ensures that the text generated by the model is more reliable and accurate.
What makes CRAG special is that it does not only rely on a fixed database but can also obtain more real-time information through web searches. This means that even if the initial retrieval is erroneous, CRAG can find the correct answers through broader channels, enhancing the accuracy of generation.
Core Idea of CRAG#
The innovation of CRAG lies in the introduction of a retrieval evaluator, which checks whether the documents retrieved from external knowledge bases are relevant to the query. This evaluator helps the model assess the quality of the retrieval results and make corresponding adjustments based on different situations.
- If the retrieval results are accurate, the evaluator identifies the most important parts, ensuring that the model only uses content that is helpful for generation. This avoids the model getting bogged down in redundant information, keeping the generated content clear and focused.
- If the retrieval results are inaccurate, the evaluator triggers a correction mechanism, initiating a broader web search. This means that the model not only relies on a limited knowledge base but can also obtain more information from the internet, thereby improving the accuracy of generation.
- If the evaluator cannot determine whether the results are accurate, it adopts a compromise strategy, combining the retrieved content with the results of web searches to generate more reliable answers.
CRAG also employs a method of "decomposition-recomposition," refining the retrieved documents. This step serves to break down lengthy documents into multiple small paragraphs, retaining only the highly relevant content to the query while discarding distracting information, ensuring that the information used by the model is concise and efficient.
Technical Process of CRAG#
In the CRAG process, the retrieved information is not directly used to generate answers but is subjected to strict screening and processing to ensure higher accuracy. The entire process can be divided into several steps:
- The retrieval evaluator checks the accuracy of the documents
Whenever a user poses a question, the system first retrieves relevant documents from the existing knowledge base. At this point, CRAG introduces a "retrieval evaluator," which scores these retrieved documents to determine their direct relevance to the question.- Relevant documents: If the evaluator deems the documents sufficiently relevant, CRAG extracts key content from them instead of passing the entire document to the generation model.
- Irrelevant documents: If the evaluator determines that these documents do not match the question, it discards them and conducts a broader web search to find more suitable information sources.
- Ambiguous situations: If the evaluator cannot clearly determine the relevance of the documents, CRAG opts for a compromise solution, combining the retrieved documents with external search results for generating answers.
- Refining document processing
For documents assessed as relevant, CRAG does not directly use the entire document to generate answers but processes them through a "decomposition-recomposition" method. It breaks the document into multiple small paragraphs, evaluating each paragraph one by one to see if it contributes to the generated content. Irrelevant or noisy information will be filtered out, leaving only key paragraphs. The purpose of this step is to ensure that the generated content is more concise and accurate. - Dynamically adjusting the generation strategy
Based on the results of document evaluation, CRAG dynamically adjusts the generation process. If the retrieval results are very reliable, the system will rely solely on this internal information to generate answers; however, if the retrieval results are unsatisfactory, the system will supplement knowledge through external web searches, and even combine both when information is uncertain. This greatly improves the accuracy and consistency of the generated content.
Advantages of CRAG#
CRAG is designed to be flexible and easy to apply. It is a plug-in method that can be easily integrated into existing RAG frameworks without requiring significant modifications to the original system. In other words, CRAG can directly enhance existing generation models, making them smarter when dealing with inaccurate information.
CRAG also exhibits excellent robustness, especially when encountering irrelevant or erroneous retrieval results. Traditional RAG methods often lead to deviations in generated results due to irrelevant information retrieved, even producing incorrect content. CRAG, through its retrieval evaluator and correction mechanism, effectively filters out this noisy information, ensuring that the generated content is more reliable. Furthermore, by introducing external web searches, CRAG expands the sources of available information, allowing it to find suitable content even when facing limited static knowledge bases.
This self-correcting mechanism allows CRAG to perform stably across various generation tasks, whether for short text responses or long-form text generation, reducing errors and providing higher quality outputs. This capability makes CRAG particularly suitable for scenarios requiring high accuracy, such as factual statements or question-answering tasks.
All of this makes CRAG a powerful tool for enhancing the accuracy of generation models, especially when facing uncertain or potentially erroneous information.
Future Work and Limitations#
Although CRAG improves the reliability of generation systems, there are still some challenges worth noting. First, CRAG relies on the accuracy of the retrieval evaluator. If the evaluator fails to correctly assess the relevance of the documents, the system may still generate inaccurate information. Therefore, future research could focus on how to further enhance the precision of the evaluator to better filter out useful content.
Additionally, an important aspect of CRAG is the introduction of external web searches, especially when retrieval results are not sufficiently relevant. While this method broadens the sources of information, it also carries the risk of introducing noise or erroneous information, as the quality of content on the internet varies. Future improvements could consider how to more effectively filter web search results to ensure that the external information introduced is sufficiently reliable.
Finally, although CRAG performs excellently across multiple tasks, its applicability in specific domains (such as highly specialized medical or legal text generation) still needs to be validated. Therefore, future research could focus on more detailed optimizations for different fields to ensure high levels of generation quality across various scenarios.