# Recursive Language Models (RLMs) # 递归语言模型 (RLMs) ![Recursive Language Models (RLMs)](https://outcomeschool.com/_next/image?url=%2Fstatic%2Fimages%2Fblog%2Frecursive-language-models.png&w=1200&q=75) In this article, we will explore **Recursive Language Models (RLMs)**, a new approach to leveraging language models to handle exceptionally large inputs that exceed the model's standard context window. 在本文中，我们将深入探讨**递归语言模型（Recursive Language Models，简称 RLMs）**。这是一种利用大语言模型（LLM）处理超大输入的新型方法，能够有效解决输入内容超出模型标准“上下文窗口”（Context Window）的限制。在大语言模型中，上下文窗口指的是模型在单次处理中能够接收和理解的最大 Token（文本单位）数量。由于传统 Transformer 架构的注意力机制计算复杂度随长度呈二次方（$O(N^2)$）增长，单次处理极长文本会带来巨大的计算资源消耗。 We will cover the following: 我们将涵盖以下内容： * What is a Recursive Language Model (RLM)? / 什么是递归语言模型（RLM）？ * Why do we need RLMs? / 为什么我们需要 RLM？ * How an RLM works / RLM 的工作原理 * How the model writes and runs code / 模型如何编写并运行代码 * Why RLMs work better / 为什么 RLM 效果更好 * Recursion inside RLMs / RLM 内部的递归机制 * How RLMs differ from simple chunking / RLM 与简单分块的区别 * Advantages of RLMs / RLM 的优势 * Limitations of RLMs / RLM 的局限性 * When to use RLMs / 何时使用 RLM * RLM vs RAG / RLM 与 RAG 的对比 * A real use case / 实际应用案例 --- ## What is a Recursive Language Model (RLM)? ## 什么是递归语言模型（RLM）？ A **Recursive Language Model (RLM)** is a framework where a language model can call another language model (or recursively call itself) on partitioned subsets of the input to progressively solve a larger problem. **递归语言模型（RLM）**是一种使用大语言模型的设计模式。在这种模式下，模型可以将庞大的输入内容拆分为较小的子部分，并通过调用另一个语言模型（或递归地调用自身）来逐步解决更大的问题。 In simple words, instead of feeding the entire document into the model at once, the system breaks the input down and processes it hierarchically. 简单来说，它并不是一次性将所有文档内容强行塞给模型，而是将输入内容拆解为更小的片段，并进行分层级的处理。 $$\text{RLM} = \text{Recursive} + \text{Language Model}$$ $$\text{RLM} = \text{递归} + \text{语言模型}$$ Here, **Recursive** means the model can call itself (or another model) repeatedly on smaller parts of the problem. **Language Model** is the LLM that understands and generates text. 在这里，**递归（Recursive）**意味着模型可以针对问题的更小部分，重复调用自身（或另一个模型）；**语言模型（Language Model）**则是负责理解和生成文本的大语言模型本身。 --- ## Why do we need RLMs? ## 为什么我们需要 RLM？ Let's say we have a very long document with 1 million tokens, and we want to ask a question about it. When we attempt to feed massive amounts of text directly into a standard language model, several major issues arise: 假设我们有一份包含 100 万个 Token（文本单位）的超长文档，并希望针对它提出一个问题。当我们尝试直接将海量文本输入到标准语言模型中时，会遇到以下几个主要问题： * **Context Window limits**: Models have a hard context window limit. They cannot accept an infinite number of tokens at once. * **上下文窗口限制**：模型具有严格的上下文窗口限制，无法在单次交互中接收无限量的 Token。 * **Context rot (Lost in the Middle)**: The quality of the response degrades as the context grows longer. The model gets confused by excessive information and often misses key details located in the middle of the prompt. This degradation is often referred to as **context rot** or the "Lost in the Middle" phenomenon. * **上下文退化（迷失在中间）**：随着上下文长度的增加，回答的质量会明显下降。模型会被过多的信息干扰，容易忽略处于提示词中间位置的关键细节。这种现象在学术界和工业界常被称为“迷失在中间”（Lost in the Middle）或**上下文退化**。 * **High latency and cost**: The execution speed of the model slows down and the financial cost of running API calls increases exponentially with context size. * **高延迟与高成本**：处理的上下文越长，模型的运行速度就越慢（延迟增加），同时运行 API 的算力成本也会呈指数级上升。 This means, simply expanding the context window does not fully solve the problem. We need a more structured approach. This is where RLMs come to the rescue. 这意味着，仅仅拓宽上下文窗口并不能从根本上解决问题。我们需要一种更具结构化的方法，而递归语言模型（RLM）正是为此而诞生。 --- ## How an RLM works ## RLM 的工作原理 In an RLM, the main model does not directly ingest the entire raw input. Instead, the full input remains as a variable inside a secure execution environment (such as a Python sandbox). The main model is given a lightweight prompt that only references this variable name and provides instructions on how to interact with it. The model then generates executable code to slice the input and delegate smaller chunks to sub-models. The results of these sub-calls are returned as concise summaries, which the main model synthesizes into the final response. 在 RLM 架构中，主模型并不直接读取完整的原始输入。相反，完整的输入会作为变量存储在一个安全的代码执行环境（例如 Python 沙箱）中。主模型接收到的只是一个轻量级的提示词（Prompt），该提示词仅包含这个变量的名称以及如何与该变量交互的指令。随后，模型会编写可执行的代码来对输入内容进行切片，并将较小的分块委托给子模型处理。这些子调用的结果会以简短摘要的形式返回，最终由主模型整合出最终答案。 The reason it is called recursive is that each sub-model call can itself act as an RLM if its allocated piece is still too large, spawning further nested calls. This closely mirrors recursion in traditional programming. 之所以被称为“递归”，是因为如果分配给某个子模型的文本块仍然太大，该子调用自身也可以作为 RLM 运行，从而产生进一步的嵌套调用。这与传统编程中的递归算法极其相似。 --- ## How the model writes and runs code ## 模型如何编写并运行代码 Let's understand this with a practical example. Suppose we have a long meeting transcript and we want to ask: *"What were the action items?"* 让我们通过一个实际例子来理解。假设我们有一份很长的会议纪要，我们想问：*“有哪些待办事项（Action Items）？”* Because language models generate text and cannot execute code directly, we wrap the model with an active runtime environment (such as a Python REPL). This environment contains a variable `context` storing the full transcript, and a function `call_llm(prompt)` that handles API calls to auxiliary models. 由于大语言模型本身只能生成文本，无法直接运行代码，因此我们需要为模型配备一个动态的运行环境（例如 Python 解释器沙箱）。该环境包含一个存储完整会议纪要的变量 `context`，以及一个用于向辅助模型发起 API 调用的函数 `call_llm(prompt)`。 We send a short prompt to the main model: 我们向主模型发送一条简短的提示词： ```text Question: What were the action items? Write Python code to answer this. The Python environment has: - A variable named `context` that stores the full transcript. - A function `call_llm(prompt)` that calls another language model. ``` **Crucially:** The full transcript is never injected into the main model's prompt. Only the variable name `context` is passed. The transcript remains isolated in the Python environment, preserving the main model's context capacity. **至关重要的一点是：** 完整的会议纪要从未被注入到主模型的提示词中，传入的仅有变量名 `context`。会议纪要始终隔离在 Python 运行环境中，从而保护了主模型的上下文容量不被撑满。 Now, the loop runs as below: 接下来，交互循环按以下步骤运行： **Step 1:** The model generates Python code as its response: **步骤 1：** 模型生成 Python 代码作为其回复： ```python turns = context.split("\n") results = [call_llm(f"Find action items in: {turn}") for turn in turns] print(results) ``` Here, the model decides to split `context` by turn and call `call_llm` on each segment. 在这里，模型自主决定按换行符拆分 `context`，并针对每个分段调用 `call_llm`。 **Step 2:** The code executes in the sandbox. Each `call_llm` call sends one segment to another language model and gets back a short answer. All answers are collected into a list. **步骤 2：** 代码在沙箱环境中运行。每一次 `call_llm` 调用都会将一个文本分段发送给另一个语言模型，并返回一个简短回答。所有的回答都会被收集到一个列表中。 **Step 3:** The list is sent back to the model. The model now sees only the short answers, not the full transcript. **步骤 3：** 收集到的列表被传回给主模型。此时，主模型看到的只有简短的中间答案，而无需接触完整的原始文本。 **Step 4:** The model reads the list and returns the final list of action items as the answer. If the answer was not clear yet, the model would write more code and the loop would continue. **步骤 4：** 模型读取该列表，并整理出最终的待办事项作为最终输出。如果此时答案仍不完整，模型会继续编写更多代码进行迭代，直到得出满意结果。 This back-and-forth between the model and the code execution environment is referred to as an **Agentic Loop** (智能体循环). In AI engineering, an Agentic Loop refers to a design pattern where an LLM is empowered to iteratively generate actions (like code execution or tool calls), observe the environment's feedback, and refine its subsequent actions until a goal is met. 模型与代码执行环境之间的这种往复交互被称为**智能体循环（Agentic Loop）**。在人工智能工程中，智能体循环是指一种设计模式：大语言模型被赋予迭代生成操作（如运行代码或调用外部工具）的能力，观察执行环境返回的反馈，并据此调整下一步的行动，直至完成既定目标。 --- ## Why RLMs work better ## 为什么 RLM 效果更好 Each sub-call processed by `call_llm` receives a highly targeted, localized chunk (e.g., just one line or turn of the meeting). Because smaller contexts contain less noise, the model can maintain high attention and accuracy without suffering from attention dilution. 被 `call_llm` 处理的每个子调用都只接收高度针对性的局部文本块（例如仅一行对话）。由于较小的上下文包含更少的噪声，模型可以保持高度的注意力和准确性，而不会遭受注意力稀释的困扰。 Additionally, the main model only sees the condensed intermediate answers, keeping its own context clean, compact, and free from "context rot." 此外，主模型最终只读取浓缩后的中间答案，这保持了自身上下文的干净、紧凑，从而彻底避免了“上下文退化”问题。 --- ## Recursion inside RLMs ## RLM 内部的递归机制 Let's delve deeper into the recursive dynamic. If a specific slice of the document is still too large for a sub-model call, that sub-model can itself act as an RLM, recursively invoking another layer of partition. 让我们更深入地探讨这种递归动态。如果文档的某个特定切片对于子模型调用而言仍然太大，该子模型自身也可以作为 RLM 运行，递归地调用下一层切片机制。 The hierarchical structure resembles a tree: 这种分层结构类似于一棵树： * **Root RLM**: Works on the full, global input context. * **根节点 RLM**：负责管理全局的整体输入任务。 * **Sub-RLMs**: Process medium-sized segments. * **子 RLM**：处理中等大小的分段。 * **Leaf nodes**: Run simple model calls on the atomic pieces of text. * **叶子节点**：在最细粒度的叶子文本片段上运行简单的模型调用。 The recursion depth is self-determined by the model based on the input scale and query complexity. 递归的深度完全由模型根据输入规模和查询复杂度自行决定。这就是递归语言模型的魅力所在。 --- ## How RLMs differ from simple chunking ## RLM 与简单分块的区别 How is this different from just splitting the input into fixed-size chunks and processing each chunk? 这与单纯将输入切分成固定大小的文本块（Chunking）并逐个处理有什么区别？ * **Simple Chunking**: Uses static, predetermined chunk sizes (e.g., fixed characters or token limits). Every chunk undergoes identical extraction rules, which can break semantic continuity (e.g., cutting a sentence in half) and lacks adaptability to the query. * **简单分块（Simple Chunking）**：使用在开发阶段预先确定的固定字符或 Token 限制。每个分块都经历相同、静态的提取规则，这可能会破坏语义的连贯性（例如将一句话从中间截断），且无法根据具体问题进行调整。 * **RLM**: Dynamically determines chunking boundaries and querying strategies on the fly. The model behaves like a human reader—skimming, skipping, or focusing deeply depending on the nature of the question. * **递归语言模型（RLM）**：能够根据当前任务动态决定分块边界和查询策略。模型的行为更像人类读者——根据问题的性质，进行略读、跳读或在特定段落进行深度聚焦。 --- ## Advantages of RLMs ## RLM 的优势 * **Handles virtually boundless inputs**: RLMs can work with data volumes far exceeding any individual model's context window. * **处理几乎无限长的输入**：RLM 可以处理远超任何单一模型上下文窗口限制的海量数据。 * **Mitigates attention degradation (context rot)**: By delegating tiny pieces to sub-calls, accuracy is maximized. * **减轻注意力退化（上下文损耗）**：通过将极小的片段委托给子调用，能够最大化保障模型回答的准确性。 * **Optimizes cost in specific scenarios**: The system avoids passing the entire massive raw context in every single turn, sending only targeted pieces. * **在特定场景下优化成本**：系统避免了在每次交互中都重复传递庞大的原始上下文，而只需发送目标片段。 * **Modular, human-like decomposition**: The model solves complex problems by breaking them down into logical, smaller sub-problems. * **模块化、类人化的任务分解能力**：模型通过将复杂的大问题拆解为合乎逻辑的微观子问题来解决任务。 --- ## Limitations of RLMs ## RLM 的局限性 * **Higher engineering complexity**: Requires a robust runtime environment, sandboxed execution, and proper tool-calling integration. * **较高的工程复杂度**：需要配置安全可靠的代码运行沙箱环境、以及完善的工具调用（Tool Calling）集成。 * **Latency overhead**: One question can trigger sequential/parallel nested API calls, which increases overall response latency. * **延迟增加**：单个问题可能会触发一系列顺序或并行的嵌套 API 调用，这会显著增加整体响应延迟。 * **Cascade error propagation**: If a leaf-level sub-call produces an erroneous summary, the error cascades up to the final synthesized response. * **级联误差传播**：如果叶子节点的子调用产生了错误摘要，该错误会向上传播并影响最终的整合答案。 * **Debugging challenges**: Because the model dynamically generates and runs code, tracking down runtime logical errors can be difficult. * **调试难度高**：由于代码是由模型动态生成并运行的，定位运行时的逻辑错误会更加困难。 --- ## When to use RLMs ## 何时使用 RLM RLMs are highly recommended for the following scenarios: 在以下场景中，强烈建议使用 RLM： * Question answering over extremely long documents or books. / 针对超长文档或整本书籍进行问答。 * Analyzing and refactoring large, multi-file codebases. / 分析和重构包含多个文件的大型代码库。 * Summarizing long-running meeting transcripts or videos. / 总结数小时的会议纪要或长视频内容。 * Performing comprehensive cross-document comparison. / 进行复杂的跨文档对比分析。 * Tasks that naturally decompose into independent, modular sub-tasks. / 任务本身可以自然拆解为相互独立、模块化的子任务。 --- ## RLM vs RAG ## RLM 与 RAG 的对比 Retrieval-Augmented Generation (RAG) is another leading framework for dealing with large datasets [1]. Let's compare their key differences: 检索增强生成（Retrieval-Augmented Generation，简称 RAG）是另一种处理大型数据集的主流框架 [1]。让我们来看看它们之间的核心区别： | Feature / 特性 | RAG | RLM | | :--- | :--- | :--- | | **Data Selection**
数据提取方式 | A search step (e.g., vector embeddings) retrieves relevant chunks first.
通过检索步骤（如向量嵌入匹配）首先筛选出相关片段。 | The model itself explores and partitions the data dynamically.
模型自主动态地探索和分割数据。 | | **Flexibility**
灵活性 | Bound by retriever accuracy and fixed indexing.
灵活性较低，高度受限于检索器的精度。 | Highly flexible; the model adapts its slicing strategy per query.
灵活性极高，模型根据不同查询自主决定处理逻辑。 | | **Code Execution**
代码执行 | None. The pipeline is static.
无代码执行能力，工作流通常是静态的。 | Yes. The model writes and runs code inside an execution sandbox.
具备代码执行能力，模型可以在沙箱中编写并运行代码。 | | **Best For**
最佳适用场景 | Question answering on static, massive knowledge bases.
针对海量静态知识库的知识问答。 | Complex, logical analysis on structured or extremely long single inputs.
针对结构化或超长单一输入的复杂逻辑分析任务。 | | **Complexity**
实现复杂度 | Relatively easier to build and deploy.
实现相对简单。 | Harder to implement due to sandbox and agentic feedback loops.
由于涉及沙箱运行和智能体反馈循环，实现较复杂。 | --- ## A real use case ## 实际应用案例：代码库安全审计 Let's look at a practical application: scanning a large software repository for security vulnerabilities. The codebase spans hundreds of files and millions of lines of code—far too large for a single prompt. 让我们来看一个实际的应用场景：扫描大型软件仓库中的安全漏洞。该代码库包含数页文件和数百万行代码，远远超出了单个 Prompt 的承载极限。 Using an RLM approach: 使用 RLM 的解决方案如下： 1. The codebase file structure is mapped and stored as a reference variable in the execution environment. 系统将整个代码库的目录结构映射并作为引用变量存储在执行沙箱中。 2. The user queries: *"Identify all potential SQL injection vulnerabilities in this project."* 用户输入查询：*“找出该项目中所有潜在的 SQL 注入漏洞。”* 3. The RLM writes Python code to traverse the file directory and filter for files containing database-related queries. RLM 编写 Python 代码遍历文件目录，筛选出包含数据库交互操作的文件。 4. For each identified file, the RLM recursively spawns a sub-call to analyze that specific file for vulnerability patterns. 针对筛选出的每个文件，RLM 递归地发起子调用，分析该特定文件中的漏洞模式。 5. The sub-calls return precise, local summaries (e.g., *"No vulnerabilities found"* or *"Potential injection at line 42"*). 子调用返回精准的本地分析摘要（例如：“未发现漏洞”或“第 42 行存在潜在注入风险”）。 6. The main RLM gathers and compiles these modular findings into a unified security report. 主 RLM 收集并汇总这些模块化的分析结果，最终整合成一份统一的安全审计报告。 This architecture guarantees that the main model is never overwhelmed by the codebase size, keeping the reasoning context clean and focused. 这种架构确保了主模型绝不会被庞大的代码库体积所压垮，从而使推理上下文始终保持干净和高度聚焦。 --- ## Summary ## 总结 To summarize, we have learned: 综上所述，我们掌握了以下内容： * A **Recursive Language Model (RLM)** recursively partitions massive inputs, executing nested model calls on smaller pieces. * **递归语言模型（RLM）**通过递归划分海量输入，在更小的文本片段上执行嵌套的模型调用。 * It solves the challenges of **context window limits** and **context rot (Lost in the Middle)**. * 它有效解决了**上下文窗口限制**和**上下文退化（迷失在中间）**的行业难题。 * The main model uses an **Agentic Loop** to write and run Python code to coordinate sub-tasks. * 主模型利用**智能体循环（Agentic Loop）**编写并运行 Python 代码，从而协同各项子任务。 * RLMs are highly effective for complex, structured, or extremely long document analysis tasks. * RLM 在处理复杂、结构化或超长文档的分析任务时表现出极强的实用性。