I’m working on a Conversational App Engine based app to generate conversational apps based on the same engine. But I’m hitting the wall of tokens limit.
As you know, GPT-4 has more max tokens, but I did not get access yet, I tried several times from March 20th to get access to GPT-4 with no luck. I tried to reach OpenAI support and employees to help in get me access , with no luck.
Is there any way to increase the tokens limit for gpt-35-turbo model, or is there any available model that has more tokens and is not Beta Limited as GPT-4?
The aim of this app (and the open-source Conversational App Engine) is to make GPT app development more accessible to more innovators.
If you have any advise it getting red of this obstacle, it will be appreciated.
Can you implement a “moving window” in the conversation? i.e. omit or truncate past messages when you approach the token limits? Even though the chat.openai.com conversations can go on endlessly, the earlier content in the chat is eventually “forgotten.”
The Conversational App Engine alowes developers to prototype gpt based apps as js class, just by implementing predefined methods that 1) define the initial prompts of the conversation (othe than the user input), and, 2) define the code that will process the API response and generate the presentation view of the response. The app will not deal with the chat management, nor communicating with chat API.
This is a simple thing to do for developers with a knowledge in prompt engineering. The project has step by step instructions about this.
The result from one example app included in the project will look like:
The idea of the Conversational App Creator app is to allow non-developers to implement their ideas, by utilizing LLM to ‘understand’ the idea of the app and generate the needed JS class.
The initial prompt of the app have a detailed instructions about generating the initial prompts and the response processing code, in addition to examples on the app classes.
Considering this, in addition to the expected response tokens, 4k of tokens will not be enough to experiment different prompts or to fit with such a need.
I thought about generating the parts of the app separately, but this could lead to lose the context.
I hope I put more context about the chalange that I’ll win .
I agree, I’m working on this direction and getting knowledge in advanced prompting techniques, mainly I’m reading Learn Prompting course, and I think I’ll benefit from the new courses of deeplearning.ai that I started to watch.
But I think advanced models like GTP4 will need less prompt tokens to achieve the same results as less advanced models.
I’m planning to contribute this to the open-source community, so it can be used to build consumer ready app generators on different platforms. So, I’ll not pay a lot if the cost of innovating new apps will be distributed to a lot of providers.
I totally agree, I’m trying to gain the knowledge to be able to take the right decisions about the design needed to support different prompting techniques in the Conversational App Engine itself.
I have not reached Langchain yet, it is part of the courses that I’m learning from and mentioned on my previous reply.
I’ll consider pushing the implementation of Conversation App Creator app when it reaches a decent reliability and accept the contributions to enhance it.
Google assistant notified me about this video talking about LLM as Tool Makers, which is a paper that has been published recently (Actually it was published on the same day I stared this topic )
It talks about the same concept of letting LLM creates tools that can be run on LLM.
It shows that GPT-4 can be used as Tool Maker.
A quick update on this topic. I started to use gpt-3.5-turbo-16k-0613 model with this app, it showed good performance with less cost.
I took further steps by generating a sample user input and testing the generated class at runtime and displaying all that to the user. Also I moved the generated code to a separate tab.
Currently I’m working on enhancing the accuracy of dealing with modification requests from the user. Once I’m done, I’ll publish this app.
.
