It can be frustrating, I know. ChatGPT and LLM-based systems can be flaky, surprisingly stupid at times, and genuinely seem to get lost in long sessions, becoming dumber and making more mistakes as things go off the rails.
That being said, ChatGPT is already a useful tool in its current state and is only destined to improve as issues are found, diagnosed, and capped off. I use it daily now, and it is a hugely useful assistant, dumb as it is, for coding. It does the grunt work typing, has knowledge of APIs that exceeds my own, and despite the frustration and missteps, it can produce working ‘hacks’ that I find useful.
Getting good output from ChatGPT is a skill that must be developed. It is possible to get something useful from it, but right now you have to be sufficiently expert in the area to recognize when it has gotten it wrong and correct it.
In areas where I have tried ChatGPT and other AI systems, one of the most frustrating things is the fact that the human data it trained on is poor. The majority of human-written source code is simply dreadful, filled with very poor habits, mistakes, and often simply does not work. The wide net that the training casts gives it many advantages and genuinely useful emergent properties. However, it is also indiscriminate when it comes to my area of expertise. Particularly vexing is the parroting of arguments in favor of things like global variables, multiple points of exit, sloppy constructs, etc.
I recently wrote this on Reddit:
“As with other languages, lots of C source, perhaps most, is badly written. Old school rules of thumb are still ignored by most programmers partially because they don’t know them, but also crazily because they disagree with them. Global variables should never be used. Functions should do one thing well. Code blocks (like functions) should only have a single point of entry and a single point of exit. Resource allocations (memory is just another resource) should be explicit and deallocated essentially as a single allocate/use/deallocate construct. If you ‘get’ something, you should also ‘unget’ it. I never use assert() because it violates ‘single point’ and possibly deallocation and other cleanup. It interferes with graceful error recovery.”
There ensued, as I predicted, some arguments for when to use ‘assert()’ by people who I am pretty sure have less than my four decades of programming behind them.
For people training models: I don’t know if they express it this way, but I think some have recognized that Pareto distributions mean that about 4/5 of the training data is poor and should be stripped out. If it is not being done already, I would suggest using things like Google’s algorithms to assess the quality of data by the nature of references made to it.
Anyway, warts and all, ChatGPT is a useful tool and gets more useful as you gain skill with ChatGPT prompting and with your knowledge and skill with your craft.