# Alternatives to Assistant API **URL:** \ **Category:** API\ **Tags:** assistants-api\ **Created:** [January 16, 2024, 8:03pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772 "2024-01-16T20:03:37Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![kunal](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@kunal](https://community.openai.com/u/kunal)\ **Post date:** [January 16, 2024, 8:03pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/1 "2024-01-16T20:03:37Z")
I started building out a new project with the assistants api due to built in thread and context management but after testing the MVP, I want more granular control over when I use what model, how context is summarized, max tokens, etc. Is there a planned release for at least some of these features? If not, what is everyone else using now instead of the assistants api or langchain? Is it basically just roll your own thread and context management and then use the completions endpoint? ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [January 16, 2024, 8:25pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/2 "2024-01-16T20:25:26Z")
> [@kunal](#): > > If not, what is everyone else using now instead of the assistants api or langchain? Is it basically just roll your own thread and context management and then use the completions endpoint? Yep - especially with AI code generation, getting these systems up and running is starting to become incredibly easy. We found langchain (and even the openai libs) to be an encumberance. However, it should probably be said that some of us have been doing this for quite a while now, so we might have bliners on regarding how hard or easy things are for beginners. So it’s quite possible that starting with custom GPTs, graduating to assistants, maybe using a framework, and eventually “rolling your own” might be a reasonable path. 🤔 ---
**Author:** ![kunal](https://avatars.discourse-cdn.com/v4/letter/k/dec6dc/32.png) [@kunal](https://community.openai.com/u/kunal)\ **Post date:** [January 16, 2024, 8:38pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/3 "2024-01-16T20:38:05Z")
Appreciate it – I thought assistants would be sufficient but I’m going to move over to rolling my own thread and context management setup. Outside of langchain is there anything else I should consider or just crank out my own setup over a couple days? ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [January 16, 2024, 8:47pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/4 "2024-01-16T20:47:22Z")
Hmm - probably depends on your skill as a software developer. We found Langchain to be too restrictive, but some people seem to be getting decent results with no-code tools. Perhaps a mix could be a good option depending on what you’re trying to do. Hmm hmm hmm 🤔 I have no solid advice here other than “stay modular and don’t skip your automated tests” 😆 ---
**Author:** ![davidnorena](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/davidnorena/32/261698_2.png) [@davidnorena](https://community.openai.com/u/davidnorena)\ **Post date:** [January 23, 2024, 5:22pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/5 "2024-01-23T17:22:56Z")
What do you mean with “Langchain to be too restrictive”, can you give us an example to understand why is it restrictive ? ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [January 23, 2024, 5:45pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/6 "2024-01-23T17:45:41Z")
Hi! That’s just an opinion. I would say it feels very similar to a lot of these no-code tools. We (company) gave it a shot but found that if you want to extend something and try new things, it becomes pretty cumbersome due to a lack of easily accessible documentation, and high level of abstraction. It felt like building the necessary adapters would take more effort than building the functionality LangChain offers from scratch. Of course if you’re a diehard LangChain fan and super comfortable with the framework and it works for you, that’s great! Maybe you can teach us the way 🤠 ---
**Author:** ![anon10827405](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@anon10827405](https://community.openai.com/u/anon10827405)\ **Post date:** [January 23, 2024, 5:48pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/7 "2024-01-23T17:48:59Z")
100%. It’s powerful, but if and once you wish to improve/optimize things it becomes a massive burden. ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [January 23, 2024, 5:52pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/8 "2024-01-23T17:52:50Z")
All that said, a lot of money seems to be pushing LangChain (IBM, AWS) So it’s quite possible that you can soon earn tons of money as a consultant, fixing (or rather life-supporting) sunken-cost LangChain projects. 🤔 ---
**Author:** ![anon10827405](https://avatars.discourse-cdn.com/v4/letter/a/ed8c4c/32.png) [@anon10827405](https://community.openai.com/u/anon10827405)\ **Post date:** [January 23, 2024, 9:13pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/9 "2024-01-23T21:13:24Z")
Huh… That’s a great thought… _Scurries to create an Assistant with LangChain docs_. Just need some cardboard, a marker and I’m in business. ---
**Author:** ![SomebodySysop](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/somebodysysop/32/17613_2.png) [@SomebodySysop](https://community.openai.com/u/SomebodySysop)\ **Post date:** [January 23, 2024, 10:24pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/10 "2024-01-23T22:24:51Z")
As someone who’s been developing my own AI applications without Langchain or Python, I 2nd the motions above. Don’t get me wrong. I highly recommend Assistants API and/or Langchain when they are the right tools for the job. If they aren’t, and you have (or have access someone who has) software developer skills, then building your own is a viable option. All of these systems, no matter what language, no matter what libraries, are all essentially doing the same API calls to the models – chat completion, assistants and embeddings. EVERYBODY going through the API is using the same basic references which are documented here: [https://platform.openai.com/docs/api-reference](https://platform.openai.com/docs/api-reference) I made the commitment from the start to build myself. As it turns out, the coding was the easiest part. The hardest part was understanding how everything works. This flowchart got me started: ![Multiple PDF Chatbot Architecture 2023-05-04_11-41-45](https://us1.discourse-cdn.com/openai1/original/3X/2/3/235262243e1a3d433415ce810f4fa0dac9dc2cd1.jpeg) Certainly not everything you need to know, but a good place to start if, like I was, you don’t know anything. [![](https://us1.discourse-cdn.com/openai1/original/3X/a/c/ac23beb1f900a40a967e58fae3113318c1b0185d.jpeg "GPT-4 Tutorial: How to Chat With Multiple PDF Files (~1000 pages of Tesla's 10-K Annual Reports)") ](https://www.youtube.com/watch?v=Ix9WIZpArm0) ---
**Author:** ![petrbrzek](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/petrbrzek/32/264824_2.png) [@petrbrzek](https://community.openai.com/u/petrbrzek)\ **Post date:** [January 24, 2024, 7:46am UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/11 "2024-01-24T07:46:42Z")
I’m working on configurable assistants that use standard Chat Completion APIs in the background. I’m interested in what your expectations would be regarding configurability. The prompt and the way the conversation is summarized are clear to me. What I’m more curious about is the configurability of RAG - what aspects should be adjustable. Also, regarding the code interpreter - whether it should be possible to set which libraries are installed, and if it should support languages like Python and JavaScript. What are your expectations? ---
**Author:** ![robertb](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/robertb/32/234200_2.png) [@robertb](https://community.openai.com/u/robertb)\ **Post date:** [January 24, 2024, 6:09pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/12 "2024-01-24T18:09:56Z")
One approach is to avoid using any abstractions, except perhaps for the LLM models, such as LangChain. Then, write all the AI logic yourself directly to the APIs, this is much simpler than many realize. This approach gives you 100% control over your own logic flow. Additionally, you can check out our open-source Policy Synth agent class library, where, due to it’s polymorphic nature, you have full control over the level of abstraction you want to use, if any: [GitHub - CitizensFoundation/policy-synth: Policy Synth is a Typescript class-based library for creating AI agent logic flows, API's and state of the art realtime web applications. The drive behind the project is to help governments and citizens make better decisions together by seamlessly integrating collective and artificial intelligence.](https://github.com/CitizensFoundation/policy-synth) ---
**Author:** ![jlvanhulst](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jlvanhulst/32/195305_2.png) [@jlvanhulst](https://community.openai.com/u/jlvanhulst)\ **Post date:** [January 24, 2024, 6:53pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/13 "2024-01-24T18:53:37Z")
I think it will be very interesting to see how thinking and implementations about this will develop. A lot of questions on the forum are about ‘flow control’ and ‘logic’ (including function calling) - and I find my self wondering about design decisions around it every day. What part of the ‘logic’ do I build - vs ‘prompt’. And when I try to ‘prompt’ and it doesn’t work the way I expect (always) - do I fall back to ‘build’. Right now I feel my role becoming more of an orchestrator - trying to create reliable connections - and explain the ways of interacting between systems. I think coders like ‘control’ and code can be buggy (but always seems 'fixable) wheres GPT’s can seem ‘fuzzy’ - but often it really is that we’re actually not very good at writing clear instructions. (Which is funny to some extent - in actual code we have to be super precise and detailed - in prompts we often seem to think that ‘a few words’ should be enough. Interesting times! p.s. I love the Assistants and run 12 of them across Email, chat bot and internal applications, connecting with Salesforce, Pitchbook and a few others. ---
**Author:** ![sreekanth850](https://avatars.discourse-cdn.com/v4/letter/s/c89c15/32.png) [@sreekanth850](https://community.openai.com/u/sreekanth850)\ **Post date:** [January 25, 2024, 5:15am UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/14 "2024-01-25T05:15:22Z")
Problem is the cost. A managed vector db will cost you minimum of 70-100 usd per month to start with, Then the cost of deploying LLM or the token cost if you use Open AI or any such platform. Then another issue will be latency for all this service. ---
**Author:** ![SomebodySysop](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/somebodysysop/32/17613_2.png) [@SomebodySysop](https://community.openai.com/u/SomebodySysop)\ **Post date:** [January 25, 2024, 7:20am UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/15 "2024-01-25T07:20:02Z")
> [@sreekanth850](#): > > Problem is the cost. A managed vector db will cost you minimum of 70-100 usd per month to start with If you go with PineCone. If you go with Weaviate, that cost drops to $25 a month for their SaaS. If you download their open source package and install locally, that cost drops to 0 per month. ---
**Author:** ![SomebodySysop](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/somebodysysop/32/17613_2.png) [@SomebodySysop](https://community.openai.com/u/SomebodySysop)\ **Post date:** [January 25, 2024, 7:28am UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/16 "2024-01-25T07:28:17Z")
> [@jlvanhulst](#): > > I think coders like ‘control’ and code can be buggy (but always seems 'fixable) wheres GPT’s can seem ‘fuzzy’ - but often it really is that we’re actually not very good at writing clear instructions. (Which is funny to some extent - in actual code we have to be super precise and detailed - in prompts we often seem to think that ‘a few words’ should be enough. Hate to break this to you, Pal, but GPTs are… coded as well. Just because you don’t have to code to use them doesn’t mean they aren’t computer programs like everything else. In fact, every no-code, chat with your pdf, LangChain, whatchamacallit out there is, in the end, developed with CODE. Which means, they are as susceptible to “buggy” as anything else. > [@jlvanhulst](#): > > p.s. I love the Assistants and run 12 of them across Email, chat bot and internal applications, connecting with Salesforce, Pitchbook and a few others. All that cool no-code stuff you’re using? Somebody wrote the code so you didn’t need to. ---
**Author:** ![jlvanhulst](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/jlvanhulst/32/195305_2.png) [@jlvanhulst](https://community.openai.com/u/jlvanhulst)\ **Post date:** [January 25, 2024, 1:34pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/17 "2024-01-25T13:34:31Z")
I’m fully aware! I am talking about the dilemma that ‘we’ collectively are faced with with these GPTS. What to ‘code’ and what to ‘prompt’. And yes, under everything is code. And writing a prompt is writing code - in a language that we are supposed to be fluent in, but the interpeter running our prompt code, unlike with ‘regular code’ will return slightly (or sometimes -wildly-) different results for the same prompt and the same input. ---
**Author:** ![sreekanth850](https://avatars.discourse-cdn.com/v4/letter/s/c89c15/32.png) [@sreekanth850](https://community.openai.com/u/sreekanth850)\ **Post date:** [January 26, 2024, 10:50am UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/18 "2024-01-26T10:50:20Z")
What about the cost of setting up a low latency infrastructure for chat to work properly. I don’t know how people do, but when you go for production DB, it should be setup with redundant infrastructure and the cost involved will shoot up exponentially. Unless you have people who can setup such infrastructure and maintain it, always go with SaaS option. Now coming back to using pine cone, still you have to pay for LLM tokens additionally and need to input the whole conversation for building the context. In my opinion it will be better to optimize assistant API by splitting the threads to 4 conversation/thread and append last 2 or 3 conversation for the context. ---
**Author:** ![SomebodySysop](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/somebodysysop/32/17613_2.png) [@SomebodySysop](https://community.openai.com/u/SomebodySysop)\ **Post date:** [January 26, 2024, 11:34am UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/19 "2024-01-26T11:34:10Z")
> [@sreekanth850](#): > > What about the cost of setting up a low latency infrastructure for chat to work properly. I don’t know how people do, but when you go for production DB, it should be setup with redundant infrastructure and the cost involved will shoot up exponentially. Unless you have people who can setup such infrastructure and maintain it, always go with SaaS option. In my case, using OpenSource software for infrastructure (Drupal CMS), and on AWS EC2 instance that is duplicated daily. Our costs are low, but you do need people who can maintain it. That’s for sure. > [@sreekanth850](#): > > Now coming back to using pine cone, still you have to pay for LLM tokens This is true in any case. > [@sreekanth850](#): > > additionally and need to input the whole conversation for building the context Definitely not true – not at least with the chat completion API. I have used the Standalone Question method successfully for nearly a year now. Even made a video about it: [https://youtu.be/B5B4fF95J9s](https://youtu.be/B5B4fF95J9s) > [@sreekanth850](#): > > In my opinion it will be better to optimize assistant API by splitting the threads to 4 conversation/thread and append last 2 or 3 conversation for the context. In your case, this is true. Assistants API was made for you. My point is, it is not always the best tool for everyone. I have nearly 200,000 objects in my vector store. Not something I want to try and fit inito 20 files. ---
**Author:** ![sreekanth850](https://avatars.discourse-cdn.com/v4/letter/s/c89c15/32.png) [@sreekanth850](https://community.openai.com/u/sreekanth850)\ **Post date:** [January 26, 2024, 12:00pm UTC](https://community.openai.com/t/alternatives-to-assistant-api/589772/20 "2024-01-26T12:00:07Z")
> [@SomebodySysop](#): > > In your case, this is true. Assistants API was made for you. My point is, it is not always the best tool for everyone. I have nearly 200,000 objects in my vector store. Not something I want to try and fit inito 20 files. Yes, it may not suitable for such a complex use cases.