Hi @nathan.r !
Did you mean how to make ChatGPT generate diffs to print in a chat? That’s not quite what this thread was about, as I understood it. The problem is as follows. ChatGPT is wonderful to generate text snippets or program fragments, or even write small documents as a whole, but at some point you will want it to automatically edit files. It seems for that problem many are still looking for a viable solution.
For instance with my Co-Developer GPT engine you can make a GPT that has the capability to read and write text files in a local directory on your computer - e.g. source code. If a file is long and the GPT needs to change it then you don’t want to have it regenerate the whole file with 3 sentences changed or a method added, because that costs time and tokens and might even hit the token limit. That’s the point where it would be wonderful if ChatGPT could reliably print diffs in some format - not to print them to the user, but to write these diffs into an API that’s called from a GPT, or as a function call.
I tried various approaches how to structure the corresponding replaceInFile action :
- regular expression replacement
- unified diffs or diffs with context lines
- sed script (doesn’t work because ChatGPT isn’t aware of line numbers and adding them in the text confuses it)
- some special format where it has to give the start line and the end line of the replacement as text
but nothing worked reliably, so for now I settled to a simple search and replace where several literal search strings + their replacements are given, and if any of the search strings is not found or is not unique the engine just refuses to make the change and tells the GPT in the error message to re-read the file and reconsider that search string. That mostly works, but doesn’t feel like a really good solution.
As it is I guess the only reliable solution would be to have the GPT just describe changes whatever way it wants, and to train a lower grade fast local LLM to re-print files with those changes executed, but I don’t have the time to pull something like that off.
So it would be nice if OpenAI would consider that problem of having GPTs edit small portions of documents in an action and incorporate the solution into their training data - be that the unified diff format proposed in the beginning of this thread, or something else.
Best regards,
Hans-Peter