The optimal way to build AI Chatbots?

[info1158]

Hey everyone,

We’ve been using the Assistants API for a few months now and are generally happy with its ease of use and all the built-in features. However, with the recent latency issues, I’m starting to wonder if switching to Completions might be the way to go for a faster, more responsive chatbot.

Shoutout to the original post that got this discussion started Is the Assistants API really made to create efficient chatbots?

I’m looking to connect with anyone who’s built a complete chatbot solution—from setting up a vectorized knowledge base (we’re experimenting with Supabase) to integrating file uploads and managing conversation history. If you’ve tackled these challenges and found a way to streamline your system, I’d really appreciate hearing about your experiences and any tips you have.

Thanks in advance for sharing your insights!

[niklasohrn139]

Struggling with the same thing, would be great if anyone had some input on this

[icdev2dev]

The Assistant vs Completion debate keeps on coming up.

Which api to use in my case?

Your question was “which API to use in my case”. What I have been advocating for (and the code is open source) is to use both the Assistant API and the Chat Completion API. The rationale, in my mind, is simple. Use Assistants, Threads and Messages from Assistant API) as a persistent store of instructions and interactions with the LLMs. Use ChatCompletion to do text generation. One of the benefit is much better control over the interaction (i.e. one can ignore certain messages and focus the co…

Seeking the Best API Choice: Should I Use OpenAI's Assistant API or Chat Completion API?

Use both. See here (Switching from Assistants API to Chat Completion?) for more in-principle arguments. See here (GitHub - icdev2dev/selfet) for some implementation level details. See the video at the top of the post to see how conversations are modeled at roughly 5:20. The conversations are specialized threads (with Metadata) (Selfet -- Towards Fully Autonomous Multiple Agents)

[info1158]

@Which api to use in my case?

The rationale, in my mind, is simple. Use Assistants, Threads and Messages from Assistant API) as a persistent store of instructions and interactions with the LLMs. Use ChatCompletion to do text generation

This sounds brilliant. BUT, how do we actually do this? And do you have a live example of a chatbot thats built in this way? Would be interesting to try out.

[icdev2dev]

If you want to see how it works in practice:

Involving a review agent into the story creation process

See the two github links from implementation hints

GitHub - icdev2dev/selfet

GitHub - icdev2dev/luften

[info1158]

To give you some context, my company provides chatbot solutions for different clients. Some are E-commerce stores, other are real estate companies etc.

With risk of sounding naive, is it possible to build costumer service chatbots for different clients this way? And are these bots faster than just using Assistant API? Because that is my ambition, to build great chatbots faster, and better.

[icdev2dev]

ymmv

for me the assistants are perenially in Beta. Sometimes they work beautifully, some times they suck (pathetically slow) and some times they don’t work at all.

The chat completion is much more mature and adopted even outside of openai as a standard way of interacting with the LLMs.

If you begin to deep dive into these solutions (esp for clients), you may find it necessary to set session timeouts (with loggedIn user experience) and the ability to control costs. Implementing with chat completion with Threads, Assistants and Messages exclusively as a transactional interface ( as opposed to also inferencing), you get a vastly different control surface, imo

[jknt]

The Assistant API is great, having been using it for nearly a year now and haven’t faced any latency problem.