I’m working on a Conversational App Engine based app to generate conversational apps based on the same engine. But I’m hitting the wall of tokens limit.
As you know, GPT-4 has more max tokens, but I did not get access yet, I tried several times from March 20th to get access to GPT-4 with no luck. I tried to reach OpenAI support and employees to help in get me access , with no luck.
Is there any way to increase the tokens limit for gpt-35-turbo model, or is there any available model that has more tokens and is not Beta Limited as GPT-4?
The aim of this app (and the open-source Conversational App Engine) is to make GPT app development more accessible to more innovators.
If you have any advise it getting red of this obstacle, it will be appreciated.
1 Like
Can you implement a “moving window” in the conversation? i.e. omit or truncate past messages when you approach the token limits? Even though the chat.openai.com conversations can go on endlessly, the earlier content in the chat is eventually “forgotten.”
1 Like
Thanks for your reply,
It is the first message sent to the API (along with the setup messages). The prompt is complex with about 3K tokens.
1 Like
It sounds like you’re a little out of your depth here. Because later, when you wrote,
It becomes clear that whatever it is you’re trying to accomplish, this isn’t the right way to go about it.
I imagine most (if not all) of the logic should be moved away from the LLM.
2 Likes
Thanks @elmstedt
I think I did not get your idea.
The Conversational App Engine alowes developers to prototype gpt based apps as js class, just by implementing predefined methods that 1) define the initial prompts of the conversation (othe than the user input), and, 2) define the code that will process the API response and generate the presentation view of the response. The app will not deal with the chat management, nor communicating with chat API.
This is a simple thing to do for developers with a knowledge in prompt engineering. The project has step by step instructions about this.
The result from one example app included in the project will look like:
The idea of the Conversational App Creator app is to allow non-developers to implement their ideas, by utilizing LLM to ‘understand’ the idea of the app and generate the needed JS class.
The initial prompt of the app have a detailed instructions about generating the initial prompts and the response processing code, in addition to examples on the app classes.
Considering this, in addition to the expected response tokens, 4k of tokens will not be enough to experiment different prompts or to fit with such a need.
I thought about generating the parts of the app separately, but this could lead to lose the context.
I hope I put more context about the chalange that I’ll win 
.
1 Like
Google gpt prompt management. You’re going to need to get creative or pay $$$$ to use 4K tokens every time.
Thanks @martinrobson ,
I agree, I’m working on this direction and getting knowledge in advanced prompting techniques, mainly I’m reading Learn Prompting course, and I think I’ll benefit from the new courses of deeplearning.ai that I started to watch.
But I think advanced models like GTP4 will need less prompt tokens to achieve the same results as less advanced models.
I’m planning to contribute this to the open-source community, so it can be used to build consumer ready app generators on different platforms. So, I’ll not pay a lot if the cost of innovating new apps will be distributed to a lot of providers.
Thanks
Even if it does, you need to think about breaking this into many inferencing steps. Have you considered Langchain, for example?
