Rate_limit_exceeded in thread with file_search

API
1

Hello,

I try to use thread with file, in NodeJS with the sdk, with gpt-4o. It’s working with very small file but not with bigger filles. And the file where I have a problem is a json with only 29.1 ko.

With the function openai.beta.threads.runs.createAndPoll, in the json response, I have this error :
last_error: {
code: ‘rate_limit_exceeded’,
message: ‘Request too large for gpt-4o in organization org-*** on tokens per min (TPM): Limit 30000, Requested 31977. The input or output tokens must be reduced in order to run successfully. Visit https://platform.openai.com/account/rate-limits to learn more.’
}

But I also have

usage: { prompt_tokens: 994, completion_tokens: 63, total_tokens: 1057 }

So it’s far less 30000 token.

During my search to solve the problem, someone indicates to create a new thread each time but it’s already thtat I am doing. Or is the limit is for the total of token of all threads ?

I also try includes truncation_strategy but I don’t know where to put the variable, I don’t see any exemple code and I test in createAndPoll or threads.create but each time I have the 400 Unknown parameter error.

The actual code, creates from https://platform.openai.com/docs/assistants/tools/file-search?lang=node.js :

const assistant = await this.openai.beta.assistants.create({
            name: "Chatgpt chat " + this.uid + "-" + idx,
            instructions: "",
            model: "gpt-4o",
            tools: [{ type: "file_search" }],
        });

        const fileStream = fs.createReadStream(file.path);

        // Create a vector store including our two files.
        let vectorStore = await this.openai.beta.vectorStores.create({
            name: this.uid + "-" + idx
        });
        
        console.log(vectorStore)

        console.log("1")

        await this.openai.beta.vectorStores.files.uploadAndPoll(vectorStore.id, fileStream)

        console.log("2")

        await this.openai.beta.assistants.update(assistant.id, {
            tool_resources: { file_search: { vector_store_ids: [vectorStore.id] } },
        });

        console.log("3")

        const aapl10k = await this.openai.files.create({
            file: fs.createReadStream(file.path),
            purpose: "assistants",
          });

        console.log("4")
          
        const thread = await this.openai.beta.threads.create({
            messages: [
                {
                    role: "user",
                    content: this.params.content,
                    attachments: [{ file_id: aapl10k.id, tools: [{ type: "file_search" }] }],
                },
            ],
        });

        console.log(thread)

        /*var text = await new Promise((resolve, reject) => {
            const stream = this.openai.beta.threads.runs
            .stream(thread.id, {
                assistant_id: assistant.id,
            })
            .on("textCreated", () => console.log("assistant >"))
            .on("toolCallCreated", (event) => console.log("assistant " + event.type))
            .on("messageDone", async (event) => {
                console.log("messageDone")
                if (event.content[0].type === "text") {
                    const { text } = event.content[0];
                    const { annotations } = text;
                    const citations = [];

                    let index = 0;
                    for (let annotation of annotations) {
                        text.value = text.value.replace(annotation.text, "[" + index + "]");
                        const { file_citation } = annotation;
                        if (file_citation) {
                        const citedFile = await self.openai.files.retrieve(file_citation.file_id);
                        citations.push("[" + index + "]" + citedFile.filename);
                        }
                        index++;
                    }

                    console.log(text.value);
                    resolve(text.value);
                    //console.log(citations.join("\n"));
                }
                else
                {
                    console.log(event)
                }
            })
        }).catch((error) => {
            console.error(error);
        });*/

        let run = await this.openai.beta.threads.runs.createAndPoll(
            thread.id,
            { 
              assistant_id: assistant.id
            }
          );

          if (run.status === 'completed') {
            const messages = await openai.beta.threads.messages.list(
              run.thread_id
            );
            for (const message of messages.data.reverse()) {
              console.log(`${message.role} > ${message.content[0].text.value}`);
            }
          } else {
            console.log(run.status);
            console.log(run)
          }

The stream method return any response or error so I try with few searchs the createAndPoll method.

So what do I do to solve my problem ?

I don’t need the history of the precedent messages, just I push a file with a question and I want the response.

Thanks,
Jérémie