I’ve been using mini-4o and a chat completion to narrow down the selection of possible tools for a query to 0-2 (this takes about half a second), but even with mini-4o as the model for a Run, it still takes 5+ seconds for even a simple tool call. Has anyone found a way to optimize? How long are queries typically taking in your app?
Related Topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| How can I improve response times from the OpenAI API while generating responses based on our knowledge base? | 3 | 17866 | November 9, 2023 | |
| Completion Speeds - How can we optimise speeds! URGENTLY! | 8 | 1789 | December 25, 2023 | |
| Using ChatGPT 3.5 Turbo with Langchain is excessively slow | 3 | 2798 | October 21, 2023 | |
| Function Calling is VERY slow | 6 | 1957 | July 26, 2024 | |
| GPT for Sheets is executing too slowly & running into time-out errors ... any recs? | 0 | 563 | August 29, 2023 |