如何流式传输工具调用

在流式上下文中调用工具时， 消息块 将通过 .tool_call_chunks 属性以列表形式填充 工具调用块 对象。一个 ToolCallChunk 包含可选的字符串字段：工具的 name、args 和 id，还包含一个可选的整数字段 index，可用于将多个块连接在一起。字段是可选的， 因为工具调用的部分内容可能会跨不同的块进行流式传输（例如，包含参数子字符串的块对于工具名称和 ID 可能具有空值）。

由于消息块继承自其父消息类，因此一个包含工具调用块的 AIMessageChunk 也将包含 .tool_calls 和 .invalid_tool_calls 字段。 这些字段是从消息的工具调用块中尽力解析出来的。

请注意，并非所有提供商目前都支持工具调用的流式传输。在开始之前，我们先定义工具和模型。

import { z } from "zod";
import { tool } from "@langchain/core/tools";
import { ChatOpenAI } from "@langchain/openai";

const addTool = tool(
  async (input) => {
    return input.a + input.b;
  },
  {
    name: "add",
    description: "Adds a and b.",
    schema: z.object({
      a: z.number(),
      b: z.number(),
    }),
  }
);

const multiplyTool = tool(
  async (input) => {
    return input.a * input.b;
  },
  {
    name: "multiply",
    description: "Multiplies a and b.",
    schema: z.object({
      a: z.number(),
      b: z.number(),
    }),
  }
);

const tools = [addTool, multiplyTool];

const model = new ChatOpenAI({
  model: "gpt-4o",
  temperature: 0,
});

const modelWithTools = model.bindTools(tools);

现在让我们定义查询并流式传输输出：

const query = "What is 3 * 12? Also, what is 11 + 49?";

const stream = await modelWithTools.stream(query);

for await (const chunk of stream) {
  console.log(chunk.tool_call_chunks);
}

[]
[
  {
    name: 'multiply',
    args: '',
    id: 'call_MdIlJL5CAYD7iz9gTm5lwWtJ',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '{"a"',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: ': 3, ',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '"b": 1',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '2}',
    id: undefined,
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'add',
    args: '',
    id: 'call_ihL9W6ylSRlYigrohe9SClmW',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '{"a"',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: ': 11,',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: ' "b": ',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: undefined,
    args: '49}',
    id: undefined,
    index: 1,
    type: 'tool_call_chunk'
  }
]
[]
[]

请注意，添加消息块将合并其对应的工具调用块。这是 LangChain 的各种工具输出解析器支持流式处理的原理。

例如，下面我们将累积工具调用块：

import { concat } from "@langchain/core/utils/stream";

const stream = await modelWithTools.stream(query);

let gathered = undefined;

for await (const chunk of stream) {
  gathered = gathered !== undefined ? concat(gathered, chunk) : chunk;
  console.log(gathered.tool_call_chunks);
}

[]
[
  {
    name: 'multiply',
    args: '',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a"',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, ',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 1',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a"',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11,',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": ',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": 49}',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": 49}',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]
[
  {
    name: 'multiply',
    args: '{"a": 3, "b": 12}',
    id: 'call_0zGpgVz81Ew0HA4oKblG0s0a',
    index: 0,
    type: 'tool_call_chunk'
  },
  {
    name: 'add',
    args: '{"a": 11, "b": 49}',
    id: 'call_ufY7lDSeCQwWbdq1XQQ2PBHR',
    index: 1,
    type: 'tool_call_chunk'
  }
]

最后，我们可以看到最终聚合的工具调用块包含了完全收集的原始字符串值：

console.log(typeof gathered.tool_call_chunks[0].args);

string

我们还可以看到完全解析的工具调用作为对象出现在末尾：

console.log(typeof gathered.tool_calls[0].args);

object