Wondering for thoughts on alternative prompting here. The approach in general works well, but this isn’t.
I’m making a 4o API request with prompt as follows. Although it’s suggested to use just one, I’ve found temperature 0.2 and top_p 0.1 to be good in general, however with those, using only temperature, the defaults and other variants, I’m getting bad results. Despite using response_format “json_object” in the api request, sometimes the result is also invalid JSON.
Prompt
You transform code given within a <code> tag as asked. You preserve indentation. Always respond only with valid JSON containing an ‘edit’ property with the changed code as a string value.In the following code, please add _() localisation macro.
<code> wxLogDebug(wxString("EXCLUDE " + line + “\n”).c_str());</code>
Response:
{
“edit”: " wxLogDebug(wxString(_(" , "EXCLUDE " : " + line + " , “\n” : “)).c_str());”
}
This is valid JSON, but parts of the original text have become keys in the result, so the edit property has only a partial value. In another result, it repeated the JSON line a huge number of times, giving invalid JSON of several thousand tokens. In general it performs well, and I was impressed that giving it code with a mix of translatable strings and strings with control sequences, it decided only to apply the macro to sensible strings, but this specific example is always a fail in one way or another.