功能
流式响应
流式响应让你在模型生成内容时实时获取，而不是等待完整响应。
为什么使用流式
| 场景 | 非流式 | 流式 |
|---|---|---|
| 用户体验 | 等待全部生成后显示 | 逐字显示，感觉更快 |
| 长响应 | 可能超时 | 持续接收，不超时 |
| 首字节时间 | 完整响应时间 | 毫秒级 |
基础用法
TypeScript
import OpenAI from 'openai';
const client = new OpenAI({
baseURL: 'https://api.byoks.com/v1',
apiKey: 'your-byoks-key',
});
const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: [{ role: 'user', content: '写一首诗' }],
stream: true,
});
for await (const chunk of stream) {
const content = chunk.choices[0]?.delta?.content || '';
process.stdout.write(content);
}Python
from openai import OpenAI
client = OpenAI(
base_url="https://api.byoks.com/v1",
api_key="your-byoks-key",
)
stream = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "写一首诗"}],
stream=True,
)
for chunk in stream:
content = chunk.choices[0].delta.content or ""
print(content, end="", flush=True)cURL
curl https://api.byoks.com/v1/chat/completions \
-H "Authorization: Bearer your-byoks-key" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "写一首诗"}],
"stream": true
}'流式事件格式
流式响应使用 Server-Sent Events (SSE) 格式：
data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"你"},"index":0}]}
data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"好"},"index":0}]}
data: {"id":"chatcmpl-xxx","choices":[{"delta":{"content":"！"},"index":0}]}
data: [DONE]事件结构
interface StreamChunk {
id: string;
object: 'chat.completion.chunk';
created: number;
model: string;
choices: [{
index: number;
delta: {
role?: 'assistant';
content?: string;
};
finish_reason: null | 'stop' | 'length';
}];
}处理完成信号
for await (const chunk of stream) {
const choice = chunk.choices[0];
if (choice.finish_reason === 'stop') {
console.log('\n\n生成完成');
break;
}
if (choice.finish_reason === 'length') {
console.log('\n\n达到最大长度');
break;
}
process.stdout.write(choice.delta?.content || '');
}获取 Token 用量
流式响应的最后一个 chunk 包含用量信息：
let usage = null;
for await (const chunk of stream) {
if (chunk.usage) {
usage = chunk.usage;
}
// ... 处理内容
}
console.log('Token 用量:', usage);
// { prompt_tokens: 10, completion_tokens: 50, total_tokens: 60 }需要在请求中设置 stream_options: { include_usage: true }
const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: [...],
stream: true,
stream_options: { include_usage: true },
});错误处理
try {
const stream = await client.chat.completions.create({
model: 'gpt-4o',
messages: [...],
stream: true,
});
for await (const chunk of stream) {
// 处理 chunk
}
} catch (error) {
if (error instanceof OpenAI.APIError) {
console.error('API 错误:', error.message);
} else {
console.error('网络错误:', error);
}
}跨 Provider 一致性
BYOKS 确保所有 Provider 的流式响应格式一致：
// 无论使用哪个 Provider，处理方式相同
const models = ['gpt-4o', 'claude-3-5-sonnet-20241022', 'gemini-1.5-pro'];
for (const model of models) {
const stream = await client.chat.completions.create({
model,
messages: [{ role: 'user', content: '你好' }],
stream: true,
});
// 处理代码完全相同
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content || '');
}
}