Nextjs problem when streaming

API Bugs
,
1

Env: nodejs/nextjs
Openai sdk version: 4.48.3
Im having a strange behavior in my deployed nextjs application when calling an assistant and streaming. I have both api key and id setted, and locally works like a charm, even logged the env variables because I was suspecting a bug in those but I discarded since the chat.completions works also with streaming.
Its strange because the streaming works well locally, but when deploying this to vercel, it says:

69-3bad329a284508d3.js:1 Error consuming stream: 
SyntaxError: Unterminated string in JSON at position 30 (line 1 column 31)

The error and the line/column is always the same.

My code is as follows:

import { StreamingTextResponse } from "ai"
import OpenAI from "openai"

export async function POST(request: Request) {
  const json = await request.json()
  const { messages, threadId } = json
  const messageText = messages.length > 0 ? messages.pop().message.content : ""

  if (!messageText) {
    return new Response(JSON.stringify({ message: "No message provided." }), {
      status: 500
    })
  }

  try {
    const profile = await getServerProfile()

    checkApiKey(profile.openai_api_key, "OpenAI")

    const openai = new OpenAI({
      apiKey: profile.openai_api_key || "",
      organization: profile.openai_organization_id
    })

    // Retrieve the assistant.
    const assistant = await openai.beta.assistants.retrieve(
      process.env.ASSISTANT_ID || ""
    )

    console.log(process.env.ASSISTANT_ID)
    console.log(profile.openai_api_key)

    // Create or Retrieve the thread.
    let thread

    if (threadId) {
      thread = await openai.beta.threads.retrieve(threadId)
    } else {
      thread = await openai.beta.threads.create({})
    }

    // Create the message.
    await openai.beta.threads.messages.create(thread.id, {
      role: "user",
      content: messageText
    })

    const run = openai.beta.threads.runs.stream(thread.id, {
      assistant_id: assistant.id,
      stream: true
    })

    return new StreamingTextResponse(run.toReadableStream())
  } catch (error: any) {
    const errorCode = error.status || 500
    return new Response(JSON.stringify({ message: error }), {
      status: errorCode
    })
  }
}

I would be glad to give more details! This is all I have right now.

Thank you in advance.

2 Likes
2

I may have a solution for you with the sdk I created here!