如何从问答链中进行流式传输
预备知识
本指南假定您熟悉以下内容：
在问答应用中，向用户展示用于生成答案的来源信息通常非常重要。最简单的方法是让链在每次生成时返回检索到的文档。
我们将使用 Lilian Weng 的 LLM 驱动的自主代理 博客文章作为本笔记本的检索内容。
环境配置
依赖项
在本次演练中，我们将使用 OpenAI 的聊天模型和嵌入模型以及一个 Memory 向量存储。但这里展示的所有内容均适用于任何 ChatModel 或 LLM、Embeddings、以及 VectorStore 或 Retriever。
我们将使用以下软件包：
npm install --save langchain @langchain/openai cheerio
我们需要设置环境变量 OPENAI_API_KEY：
export OPENAI_API_KEY=YOUR_KEY
LangSmith
使用 LangChain 构建的许多应用程序将包含多个步骤，并多次调用 LLM。随着这些应用程序变得越来越复杂，能够检查链或代理内部确切发生了什么变得至关重要。要做到这一点，最佳方式是使用LangSmith。
请注意，LangSmith 并不是必需的，但它非常有帮助。如果确实想要使用 LangSmith，在通过上面的链接注册后，请确保设置你的环境变量以开始记录追踪信息：
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=你的密钥
# 如果你不在无服务器环境中，可通过以下命令减少追踪延迟
# export LANGCHAIN_CALLBACKS_BACKGROUND=true
带来源的链
这是我们在 Lilian Weng 的LLM 驱动自主代理博客文章中构建的问答应用，其中包含了返回来源指南中的相关内容：
import "cheerio";
import { CheerioWebBaseLoader } from "@langchain/community/document_loaders/web/cheerio";
import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
import { MemoryVectorStore } from "langchain/vectorstores/memory";
import { OpenAIEmbeddings, ChatOpenAI } from "@langchain/openai";
import { pull } from "langchain/hub";
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { formatDocumentsAsString } from "langchain/util/document";
import {
RunnableSequence,
RunnablePassthrough,
RunnableMap,
} from "@langchain/core/runnables";
import { StringOutputParser } from "@langchain/core/output_parsers";
const loader = new CheerioWebBaseLoader(
"https://lilianweng.github.io/posts/2023-06-23-agent/"
);
const docs = await loader.load();
const textSplitter = new RecursiveCharacterTextSplitter({
chunkSize: 1000,
chunkOverlap: 200,
});
const splits = await textSplitter.splitDocuments(docs);
const vectorStore = await MemoryVectorStore.fromDocuments(
splits,
new OpenAIEmbeddings()
);
// Retrieve and generate using the relevant snippets of the blog.
const retriever = vectorStore.asRetriever();
const prompt = await pull<ChatPromptTemplate>("rlm/rag-prompt");
const llm = new ChatOpenAI({ model: "gpt-3.5-turbo", temperature: 0 });
const ragChainFromDocs = RunnableSequence.from([
RunnablePassthrough.assign({
context: (input) => formatDocumentsAsString(input.context),
}),
prompt,
llm,
new StringOutputParser(),
]);
let ragChainWithSource = new RunnableMap({
steps: { context: retriever, question: new RunnablePassthrough() },
});
ragChainWithSource = ragChainWithSource.assign({ answer: ragChainFromDocs });
await ragChainWithSource.invoke("What is Task Decomposition");
{
question: "What is Task Decomposition",
context: [
Document {
pageContent: "Fig. 1. Overview of a LLM-powered autonomous agent system.\n" +
"Component One: Planning#\n" +
"A complicated ta"... 898 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
},
Document {
pageContent: 'Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are'... 887 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
},
Document {
pageContent: "Agent System Overview\n" +
" \n" +
" Component One: Planning\n" +
" "... 850 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
},
Document {
pageContent: "Resources:\n" +
"1. Internet access for searches and information gathering.\n" +
"2. Long Term memory management"... 456 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
}
],
answer: "Task decomposition is a technique used to break down complex tasks into smaller and simpler steps fo"... 230 more characters
}
让我们看看这个提示词的实际样子。你也可以在 LangChain 提示词中心 查看它：
console.log(prompt.promptMessages.map((msg) => msg.prompt.template).join("\n"));
You are an assistant for question-answering tasks. Use the following pieces of retrieved context to answer the question. If you don't know the answer, just say that you don't know. Use three sentences maximum and keep the answer concise.
Question: {question}
Context: {context}
Answer:
流式传输最终输出
通过 LCEL，我们可以以流的形式传输生成的输出：
for await (const chunk of await ragChainWithSource.stream(
"What is task decomposition?"
)) {
console.log(chunk);
}
{ question: "What is task decomposition?" }
{
context: [
Document {
pageContent: "Fig. 1. Overview of a LLM-powered autonomous agent system.\n" +
"Component One: Planning#\n" +
"A complicated ta"... 898 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
},
Document {
pageContent: 'Task decomposition can be done (1) by LLM with simple prompting like "Steps for XYZ.\\n1.", "What are'... 887 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
},
Document {
pageContent: "Agent System Overview\n" +
" \n" +
" Component One: Planning\n" +
" "... 850 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
},
Document {
pageContent: "(3) Task execution: Expert models execute on the specific tasks and log results.\n" +
"Instruction:\n" +
"\n" +
"With "... 539 more characters,
metadata: {
source: "https://lilianweng.github.io/posts/2023-06-23-agent/",
loc: { lines: [Object] }
}
}
]
}
{ answer: "" }
{ answer: "Task" }
{ answer: " decomposition" }
{ answer: " is" }
{ answer: " a" }
{ answer: " technique" }
{ answer: " used" }
{ answer: " to" }
{ answer: " break" }
{ answer: " down" }
{ answer: " complex" }
{ answer: " tasks" }
{ answer: " into" }
{ answer: " smaller" }
{ answer: " and" }
{ answer: " simpler" }
{ answer: " steps" }
{ answer: "." }
{ answer: " It" }
{ answer: " can" }
{ answer: " be" }
{ answer: " done" }
{ answer: " through" }
{ answer: " various" }
{ answer: " methods" }
{ answer: " such" }
{ answer: " as" }
{ answer: " using" }
{ answer: " prompting" }
{ answer: " techniques" }
{ answer: "," }
{ answer: " task" }
{ answer: "-specific" }
{ answer: " instructions" }
{ answer: "," }
{ answer: " or" }
{ answer: " human" }
{ answer: " inputs" }
{ answer: "." }
{ answer: " Another" }
{ answer: " approach" }
{ answer: " involves" }
{ answer: " outsourcing" }
{ answer: " the" }
{ answer: " planning" }
{ answer: " step" }
{ answer: " to" }
{ answer: " an" }
{ answer: " external" }
{ answer: " classical" }
{ answer: " planner" }
{ answer: "." }
{ answer: "" }
我们可以在返回流的同时添加一些逻辑来编译它：
const output = {};
let currentKey: string | null = null;
for await (const chunk of await ragChainWithSource.stream(
"What is task decomposition?"
)) {
for (const key of Object.keys(chunk)) {
if (output[key] === undefined) {
output[key] = chunk[key];
} else {
output[key] += chunk[key];
}
if (key !== currentKey) {
console.log(`\n\n${key}: ${JSON.stringify(chunk[key])}`);
} else {
console.log(chunk[key]);
}
currentKey = key;
}
}
question: "What is task decomposition?"
context: [{"pageContent":"Fig. 1. Overview of a LLM-powered autonomous agent system.\nComponent One: Planning#\nA complicated task usually involves many steps. An agent needs to know what they are and plan ahead.\nTask Decomposition#\nChain of thought (CoT; Wei et al. 2022) has become a standard prompting technique for enhancing model performance on complex tasks. The model is instructed to “think step by step” to utilize more test-time computation to decompose hard tasks into smaller and simpler steps. CoT transforms big tasks into multiple manageable tasks and shed lights into an interpretation of the model’s thinking process.\nTree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning possibilities at each step. It first decomposes the problem into multiple thought steps and generates multiple thoughts per step, creating a tree structure. The search process can be BFS (breadth-first search) or DFS (depth-first search) with each state evaluated by a classifier (via a prompt) or majority vote.","metadata":{"source":"https://lilianweng.github.io/posts/2023-06-23-agent/","loc":{"lines":{"from":176,"to":181}}}},{"pageContent":"Task decomposition can be done (1) by LLM with simple prompting like \"Steps for XYZ.\\n1.\", \"What are the subgoals for achieving XYZ?\", (2) by using task-specific instructions; e.g. \"Write a story outline.\" for writing a novel, or (3) with human inputs.\nAnother quite distinct approach, LLM+P (Liu et al. 2023), involves relying on an external classical planner to do long-horizon planning. This approach utilizes the Planning Domain Definition Language (PDDL) as an intermediate interface to describe the planning problem. In this process, LLM (1) translates the problem into “Problem PDDL”, then (2) requests a classical planner to generate a PDDL plan based on an existing “Domain PDDL”, and finally (3) translates the PDDL plan back into natural language. Essentially, the planning step is outsourced to an external tool, assuming the availability of domain-specific PDDL and a suitable planner which is common in certain robotic setups but not in many other domains.\nSelf-Reflection#","metadata":{"source":"https://lilianweng.github.io/posts/2023-06-23-agent/","loc":{"lines":{"from":182,"to":184}}}},{"pageContent":"Agent System Overview\n \n Component One: Planning\n \n \n Task Decomposition\n \n Self-Reflection\n \n \n Component Two: Memory\n \n \n Types of Memory\n \n Maximum Inner Product Search (MIPS)\n \n \n Component Three: Tool Use\n \n Case Studies\n \n \n Scientific Discovery Agent\n \n Generative Agents Simulation\n \n Proof-of-Concept Examples\n \n \n Challenges\n \n Citation\n \n References","metadata":{"source":"https://lilianweng.github.io/posts/2023-06-23-agent/","loc":{"lines":{"from":112,"to":146}}}},{"pageContent":"(3) Task execution: Expert models execute on the specific tasks and log results.\nInstruction:\n\nWith the input and the inference results, the AI assistant needs to describe the process and results. The previous stages can be formed as - User Input: {{ User Input }}, Task Planning: {{ Tasks }}, Model Selection: {{ Model Assignment }}, Task Execution: {{ Predictions }}. You must first answer the user's request in a straightforward manner. Then describe the task process and show your analysis and model inference results to the user in the first person. If inference results contain a file path, must tell the user the complete file path.","metadata":{"source":"https://lilianweng.github.io/posts/2023-06-23-agent/","loc":{"lines":{"from":277,"to":280}}}}]
answer: ""
Task
decomposition
is
a
technique
used
to
break
down
complex
tasks
into
smaller
and
simpler
steps
.
It
can
be
done
through
various
methods
such
as
using
prompting
techniques
,
task
-specific
instructions
,
or
human
inputs
.
Another
approach
involves
outsourcing
the
planning
step
to
an
external
classical
planner
.
"answer"
下一步
你现在已经了解了如何从问答链中流式传输响应。
接下来，查看一些关于 RAG 的其他操作指南，例如如何添加聊天历史记录。