Hey folks!
I’m also looking for ways of instructing GPT to write a precise Diff file.
I posted the idea in a new thread:
Cheers!
1 Like
Concerning the application of GPT generated diffs on codebase, this projects solves this perfectly: GitHub - paul-gauthier/aider: aider is AI pair programming in your terminal
In my experience Chat is pretty good with a few thousand words of English but is only good with about 20 to maybe as much as 30 lines of code depending on the language and the verbosity of the tasks. It is also better at writing new code than analyzing old.
It can do much better with code to perform secondary tasks like checking it for mistakes, or spin up a unit test to match.
But IMO the future will be to change coding to match how LLMs works which means changing how software is designed and written. I have started on this and it appears promising.
I don’t think it is capable of making automatic coding changes to a code file but it probably is capable of things like generating a unit tests in an automated way.
Over a year later, and sadly, ChatGPT is still quite bad at generating diffs. Critically, it seems incapable of rendering hunk headers correctly.
For example, it knows the hunk header stores the number of context lines. It knows it has, say, 5 lines of context. But for some reason, it doesn’t enter the correct number of context lines into the hunk header.
If I walk ChatGPT through each step and apply the Socratic method, then it very slowly gets it, but it immediately forgets everything if I started a new problem, and has to be walked through it. every. single. time.
Its default behavior is to just mimic diff output. Not truly generate it.
Hey I’ve actually got pretty well working solution to this problem. It took a lot of custom parsing and involves generating the diff after creating a patch from the ai response.
It works quite reliably on files with even thousands of lines of code, but optimally you use it on 1 file of context at a time.
The trick is really to forget about the ai knowing why lines it’s adding and removing, and focus on the chunks of code it wants to change.
I can’t include links, but it’s built into my free app on TestFlight - Repo Prompt.