Looks like my fears were justified.
Many of the Gemini|Bard examples in this video reflect my experience to date with GPT-3.5-Turbo-16K
codie, that is my experience as well . as long as i give chatgpt very specific instructions, chtgpt can write the code, although chtagpt has trouble rendering mathjax no matter what the situation. It seems to me that chatgpt could be easily modified to check its own outptut code. just run the output code on a specified development environment. Loop again on an error message. Then no more hallucinogenic code outputs from chatgpt. Does everyone know this and i am just mr. obvious?
1 Like
Hey kjordan, isnt the biggest limitation of Gemini (like by gazillions of miles) is that you canāt upload your own knowledge files to train the gpt?
Not super impressed with Gemini
Grok is much funnier. (also much less censored - eg it will tell you how to end life)
GPT-4 is still my goto for serious work.
(mostly python, esp misc libraries like arxiv, and processing arxiv/semantic-scholar papers)
1 Like
Hey Gary,
Yeah that is a good way and that is essentially what both the code interpreter and the plugin functionality does in GPT. Other people have implemented their own versions of this as well for different types of clients.
I think you are right on this point, there are a lot of OpenAI fans in this community, Iād say borderline cargo cult, but yeah for the most part there is missing functionality and that seems to be the biggest weakness. I imagine that will disappear pretty quickly.
The functionality that OpenAI has baked into thereās is not hard to replicated and there are already opensource versions of each of the aspects, aside from the models themselves, already available. My thinking is that there will be a period of 1 maybe 2 months of experimentation and then youāll see people porting over their projects just to say they did.
I know thatās exactly what Iāll be doing xD. Whether you are looking for clients or investors, people just want to know that you are capable of choosing the right technology and you can only do that if youāve tried them.
Iād say Gemini is not super impressive because they overhyped and Iāve already built a strong habit of talking to GPT in a style that gets good results. I imagine that will be less important over time. But Iāve noticed you just have to talk to Gemini differently.
Weāll see if this is just the chat interface or the model come the 13th when the API is available.
Ah, 13th - great, tnx, hadnāt been following that closely.
āIāve already built a strong habit of talking to GPT in a style that gets good results.ā
Have you ever tried DSPy? Might make it easier to port prompts?
(Havenāt tried it, reluctant to go through the learning curve costā¦)
No I havenāt, didnāt even know about it. But, Iām in the middle of testing out different AI skills/templates for a project I am working on. Maybe Iāll give it a whirl.
I know this is nitpicky, but i dont particularly like their UI or how it spits out coding responses. Something about the short text and then the code boxes within GPT really makes it much better. and Gemini was a bit too wordy for me when i asked it coding questions.
So iād just like to chime in and second the notion that I also tried it for a coding problem and didnāt get great results.
1 Like
Thanks for your reply, codie. " that is essentially what both the code interpreter and the plugin functionality does in GPT." Forgive my ignorance, but how are a ācode interpreterā or a āpluginā accessed on chatgpt? A month ago i tried the OpenAI Codex product (or whatever its called) and I quickly abandoned Codex because it was vastly inferior to plain old chatgpt. With plain old chatgpt, I just query, copy, paste, and run. Often I have to repeat the cycle a few times due to chatgpt hallucinations. So if you can give me some hints on where I can find some of these tools to automate my query, copy, paste, run procedure, I would sure appreciate it. Query, copy, paste, run works very well for me in at least as many as vscode, xcode, matlab, pycharm.
Chat GPT4 has the ability to create and run code. People were making plugins that did this, but then OpenAI ended up banning them because it was āunsafeā. Then eventually they made their own anyways. It does well with simple well known tasks like game of life. But there is a runtime and resource limit.
Analyze (Create the Code)
Run the Code and Responde
Final Code for Review
You can adjust it apparently. I havenāt tried it because I thought the length was fine, but maybe you can try it out.
Bard is still stupidā¦ I use it sometimes only to find restaurants. Not for any technical stuffs
1 Like
LOL, thanks much codie. It never occurred to me to ask chatgpt to run the program! I asked chatgpt to run a mathjax rendering, which in my experience chatgpt has always had problems with, and the rendering was functionally correct. Very unusual. So the additional step of asking chatgpt to run the code that it just created does result in a significant step up in chatgpt-delivered code quality for this extremely narrow example.
1 Like
lie - to make an untrue statement with intent to deceive. I donāt think so.
2 Likes
This depends on the definition.
the concept of a lie in philosophy and ethics is nuanced. While the common understanding of a lie involves intent to deceive, some philosophical definitions are broader. They may include any statement that is not true, regardless of the speakerās intent. This can encompass statements made mistakenly, based on false beliefs, or without awareness of their falsehood. Therefore, the definition of a lie can vary depending on the ethical or philosophical perspective.
In the case of the old, Palm-2 powered Bard it could also be an issue with rushed product design. The chatbot was released under time pressure and the responsible team may have focused on other alignment issues first.
Apart from that lying is a typical issue with LLMs and a shining example is the iteration of the bing bot that started to go ballistic on itās users.
I implemented this a few months back and the results were phenomenal. Night and day. You just get big working files. Sadly, it all started misbehaving a few days ago. Waiting to see if the api sorts itself out.
Gpt4 to write the code and gpt3.5 to validate. Gold.
Early on I asked gpt4 to āImagine you are a compiler. Compile this code, run it, and show me the output and any errors.ā
And it did.
What is your Q&A use case? My Q&A works fine with davinci without chat completions. chat completions e.g. 3.5-turbo and gpt4 will slow you down for Q&A.I have compared various response times from GPT3.5 with text-davinci-003 for Q&A with sumarization and the difference on a test dedicated synology box with a single CPU and 16GB RAM is a 5 second response time difference, so 3secs vs 8 secs and with a far better answer from davinci and converting messages to the prompt for a non chat completion. So davinci is more expensive but alot faster, and 100% feasible in a realworld use case, where I found gpt3.5 not to be as the model used for Q&A. You can of course cache your results and i use ChromaDB for the similarity search first which returns quickly often 200-400ms . Thats pretty damn good and works in my live support scenario.
Thanks moonlockwood. It seemed obvious that this would become the code development path, but this idea doesnt seem to be in the popular press yet but i expect it will be soon. My first idea for creating a query, copy, paste, run loop was to have a python program run other python programs but now i am thinking some sort of unix script might work better because the unix script can run programs from the command line. do you have any suggestions for me as to what the best way to so this might be? I suppose one suggestion would be āWait two months until this a standard feature of chatgpt?ā