Introduction: Can Computers Make Decisions?#
In recent years, we have often heard the term "artificial intelligence," especially when discussing chatbots or automatic translation tools. But have you ever wondered if computers can make complex decisions like humans? For instance, where should a company build a new factory, or how should a business adjust its inventory to save costs? These questions often require us to analyze large amounts of data and make decisions based on specific circumstances.
Today, many large language models (LLMs) are quite adept at answering questions, such as asking them about the weather or helping summarize the key points of an article. However, when faced with questions that require deep thinking and multi-step decision-making, they struggle. In most cases, they can only provide answers based on the information retrieved, unable to "think" through the subsequent analytical steps like humans do. This is what we will discuss today: the PlanRAG technology, designed to help these models not only answer questions but also perform data analysis step by step like a smart decision-maker.
The innovation of PlanRAG lies in the fact that it not only relies on language models to retrieve and answer questions but also teaches the model to "plan" before answering questions, just as we need to think about "what data should I look up next" before making a decision. This planning-before-retrieval mindset enables the model to better cope with complex decision-making problems.
Next, we will delve into how PlanRAG works and what practical applications it can bring us.
What is PlanRAG?#
Imagine you need to solve a complex problem, such as deciding how to plan the production process of a factory. You wouldn't blindly start operating; instead, you would first plan the steps, clarify what information is needed, then gather data, and finally make a decision based on the analysis results. PlanRAG is a similar system, but it is applied to large language models (LLMs), enabling them to "plan first, then execute" when solving complex problems.
Typically, large language models work through a technique called "retrieval-augmented generation (RAG)," directly obtaining relevant data from external sources and then generating answers. However, PlanRAG goes a step further than traditional methods: it not only retrieves and generates simply but first allows the model to create a preliminary plan for the entire problem, clarifying what information and steps are needed, and then asks questions based on the plan to obtain data. If the retrieved data is insufficient to answer the question, the model will readjust the plan and continue querying data until a decision can be made.
For example, imagine you are playing a simulation business environment game, where you play as a merchant needing to decide in which city to set up a trading point to maximize your profits. The first step of PlanRAG is to create a plan for this problem, such as determining which trade nodes to gather data from, and then generating queries to collect profit data from these nodes. Next, the model will analyze the data to determine which node is most advantageous and make a final choice.
What makes PlanRAG special is that it doesn't just retrieve data; it continuously thinks and adjusts strategies until it finds the best solution. This combination of planning and retrieval significantly enhances the model's performance in complex decision-making.
Key Issue: Decision QA#
In many real-world scenarios, decision-making is not always straightforward. Whether in business management, logistics planning, or complex choices in daily life, it often requires extensive data analysis to find the optimal solution. Decision QA is the concept proposed in the PlanRAG paper, specifically addressing such complex decision-making problems.
For example, imagine you are playing a simulation management game, such as "Victoria 3" or "Europa Universalis IV." In the game, you need to make some important decisions, such as where to send merchants to increase trade profits or how to allocate resources most efficiently in a factory to reduce costs. To make these decisions, you need to understand various data, such as the economic development status of each city, trade flow, and even the influence of other countries.
Decision QA is designed to help solve these types of problems. The model not only needs to answer simple questions like "Where should I send the merchant?" but also needs to consider the underlying data and rules, such as the economic conditions of various cities and the influence of neighboring countries. This is different from general QA systems, which simply retrieve answers based on questions; decision QA requires the model to engage in deep thinking and data analysis to make optimal decisions.
The difficulty of decision QA lies in the fact that it is not just a simple multiple-choice question but a complex decision-making process that requires a deep understanding of data, rules, and background information. PlanRAG helps the model make better choices in these complex scenarios through planning, retrieval, and iterative adjustments.
Core Steps of PlanRAG#
What makes PlanRAG unique is that it allows large language models to rely not just on simple data queries but to solve complex decision-making problems through step-by-step planning and iteration. The entire process can be divided into three key steps: planning, data retrieval, and replanning.
1. Planning: How the Model Generates a Plan#
When faced with a complex problem, we typically need to clarify what information is needed to make a decision. The PlanRAG model does the same; it first generates a plan based on the question, listing what data needs to be retrieved. This step is akin to making a shopping list before going shopping—you wouldn't go to the supermarket directly but would first consider what you need to buy.
For example, if a model needs to help a company decide where to expand into a market, it might plan to first query the competitive landscape and potential profits of various markets.
2. Data Retrieval: Asking Questions and Obtaining Data Based on the Plan#
With a clear plan in place, the model will next ask questions to the database based on this plan to retrieve relevant data. This is similar to going to the supermarket with a shopping list and finding the items you need. In this process, the model will generate corresponding queries to obtain information that can help it make decisions.
For instance, in the example of deciding where to place a merchant for maximum profit, the model will look up trade flows and potential profits for each market to see where placing the merchant yields the highest returns.
3. Replanning: Further Adjusting the Plan After Analysis#
Sometimes, the initial plan is not perfect, and the retrieved data may not fully resolve the issue. In such cases, the PlanRAG model will pause, reassess the data obtained, and decide whether to adjust the plan. This is like realizing you forgot to write down an essential item on your shopping list, so you go back to add it and continue searching.
This replanning process is crucial because it ensures that the model can flexibly adjust its analytical approach when facing complex problems until it finds the optimal solution.
Comparison with Other Methods: Why PlanRAG Excels#
To understand the advantages of PlanRAG, we can first look at how it differs from other common methods. Traditional methods, such as iterative retrieval-augmented generation (Iterative RAG), primarily rely on the model repeatedly retrieving information and then generating answers based on these retrieval results. This method is very effective for handling simple knowledge QA problems, as it focuses on finding information directly related to the question.
However, the problem arises when faced with more complex decision-making scenarios, such as making a series of analyses and judgments based on multiple data points; traditional methods often fall short. Iterative RAG is not adept at handling tasks that require multiple steps, involving planning and data reasoning. For example, when selecting the best trade node in a game scenario, the model not only needs to find all information related to the node but also needs to analyze this information comprehensively to make the correct decision. Existing methods often only find single pieces of information, neglecting the overall analysis and planning in the process.
The breakthrough of PlanRAG lies in its requirement for the model to "plan first." The model first outlines what data needs to be retrieved and how to analyze this data, which helps avoid random searches for information. Through repeated planning, retrieval, and adjustment, the model can gradually refine its decision-making process. This means that PlanRAG is more organized and efficient, especially in complex tasks that require a deep understanding of multiple data pieces.
Through experiments, PlanRAG has shown significant improvements in "decision QA" compared to Iterative RAG. In "location" scenarios, PlanRAG improved accuracy by 15.8%, while in "construction" scenarios, it also increased by 7.4%. These figures demonstrate its superiority in complex tasks. Specifically, PlanRAG not only identifies key data but also reasonably plans the entire decision-making process, greatly enhancing decision accuracy.
In simple terms, PlanRAG acts like a more meticulous decision-maker; it not only asks questions but also thinks about what questions it should ask and continuously adjusts its direction based on the answers, making it perform better in complex decision-making scenarios.