I am an intermediate level coder and often use the OpenAI API in Visual Code through a rudimentary Python module I created that allows me to import a basic function in a Jupyter notebook for coding assistance. I historically have just left temperature at about .7 but have recently moved that closer to .2 and have been getting much simpler code, but it rarely hallucinates non-working code.
I’d be curious to hear the community’s thoughts on adjusting the temperature, specifically as it relates to generating code or other responses that may prioritize accuracy over variability.
With open AI’s temperature it goes up to 2.0, From my experience just under one you can’t really go wrong especially if you want a little more variance, could start with 0.8. You probably only want to go closer to zero if you are doing things like lookups or other one shot like prompts.
Based on my experience, when prioritizing functional code, it’s safest to set the temperature hyperparameter to 0.
Setting it to 0 doesn’t mean the same code is always generated, but increasing the temperature typically does not yield good results in the context of code generation.
@fluxtah are you using this code? If so, what does your setup look like?
@dignity_for_all this has been my initial experience. I’m curious as to the decision to reduce the variability of the response leads better code, or if that decision crowds out more creative solutions.
@pds If you are asking if I am using it to generate code yes kind of but I have not actually used any of the generated code yet to validate its quality.
I use Chat GPT itself (the web interface) to do a lot of coding and run ideas by it and generate whole pieces of code and sometimes even use it direct after reviewing and it does a great job which makes me wonder… What temperature is Chat GPT set to and it also is that good? (but sometimes gets it wrong though easily corrected).