I’ve experimented with ChatGPT recently (I was and pretty much still am not very convinced about AI and coding, but I have had to stand corrected!)
The results where pretty surprising.
I asked ChatGPT to refactor a whole class I wrote in past (PHP), and it did that pretty well. I asked it to write a whole new class that does specific things, and boy that thing delivered pretty nice code. Initially unsafe, so I asked it to please escape outputs and sanitise inputs, and it went on doing that like it never did anything else.
Thus, intrigued, I created an API Key and started playing with the different models. I focused on code-davinci-002
And the results are catastrophic 
For example this prompt gives me either nothing (infinite request, never stops), or it says it is busy I shall retry later, or, it gives me a completely empty response!
The only way I got “some” responses was by limiting the max tokens to an unrealistic 20 or so tokens. Which gives useless, incomplete responses.
When I tried to set stream to true, instead, it delivered many responses… but all the same!
This is an example request I make:
curl --location 'https://api.openai.com/v1/completions' \
--header 'Authorization: Bearer XXX' \
--header 'OpenAI-Organization: XXX' \
--header 'Content-Type: application/json' \
--data '{
"model":"code-davinci-002",
"prompt":"Write a Class in Object Oriented PHP that outputs the main SEO Relevant metae tags for a WordPress theme. The code should be dynamic and therefore output content related to the current type of content shown (single post, archive, etc).",
"user":"XXX",
"max_tokens":1024
}'
The “best” reply I got is something along these lines (as said this was with a hard limit of tokens)
{
"id": "cmpl-XX",
"object": "text_completion",
"created": 1679480654,
"model": "code-davinci-002",
"choices": [
{
"text": "\n//\n//The class should adhere to the single responsibility principle, the",
"index": 0,
"logprobs": null,
"finish_reason": "length"
}
],
"usage": {
"prompt_tokens": 47,
"completion_tokens": 16,
"total_tokens": 63
}
}
So, it clearly seems to work, but only when I limit tokens to a hard cap?
Why? What am I doing wrong?
As ChatGPT 4 is not available in the API yet, and I think to do code-related requests the code-davinci-002 would be more adequate (?) I would like to use that, at least to understand more how it works, and what it can and cannot do.
Thanks for any inputs!
