Can confirm this model spits out random logits. Not even top_p = 1e-9 or a seed can stop it from being a random token generator.
Continuing from this investigation of seed, but with parameters as “deterministic” as it can be made.
==Response with seed 444==
In the heart of the dense jungle, a mischievous chimpanzee named Charlie had grown tired of the confines of his enclosure at the wildlife sanctuary. With a clever mind and nimble fingers, he managed to unlock the gate and slip away into the lush greenery. As the sun dipped below the horizon, the sanctuary staff realized Charlie was missing and launched a frantic search. Meanwhile, Charlie reveled in his newfound freedom, swinging effortlessly through the treetops and chattering excitedly to the other
fingerprint: fp_eeff13170a==Response with seed 444==
In the heart of the lush jungle, a mischievous chimpanzee named Charlie had grown tired of the confines of his enclosure at the wildlife sanctuary. With a clever mind and nimble fingers, he managed to unlock the gate and slip away into the dense foliage. As the sun dipped below the horizon, the sanctuary staff realized Charlie was missing and launched a frantic search. Meanwhile, Charlie reveled in his newfound freedom, swinging effortlessly through the treetops and chattering excitedly to the other animals
fingerprint: fp_eeff13170a==Response with seed 55599551==
In the heart of the lush jungle, a mischievous chimpanzee named Charlie had grown tired of the confines of his enclosure at the wildlife sanctuary. With a clever mind and nimble fingers, he managed to outsmart the caretakers and slip through a small gap in the fence. As he ventured into the dense foliage, a sense of freedom washed over him, and he reveled in the thrill of his newfound escape. With a mischievous glint in his eyes, Charlie swung from branch
fingerprint: fp_eeff13170a==Response with seed 666666==
In the heart of the lush jungle, a mischievous chimpanzee named Charlie had grown tired of the confines of his enclosure at the wildlife sanctuary. With a clever mind and nimble fingers, he managed to unlock the gate and slip away into the dense foliage. As the sun dipped below the horizon, the sanctuary staff realized Charlie was missing and launched a frantic search. Meanwhile, Charlie reveled in his newfound freedom, swinging effortlessly through the trees and chattering excitedly to the other animals he encountered
fingerprint: fp_eeff13170a
I only noted the first divergence in bold (you might not be able to see my bolding) different than any other, as all bets are off after a different token is produced.
Seed is included in the API call, but this should have no effect as there should be no randomness to the sampling simply from the top_p specification (with only the first logit fitting within a probability space of fractions of a percent) and in fact it gives no improvement having the same seed in the first two examples, which takes only six generated tokens before a top-token flip.
This new model will also produce garbage at much lower temperature settings than previous iterations, even though today’s gpt-3.5-turbo-0613 has already been degraded by continued alterations since release.
Then the simple fact that this is not a replacement for the 16k model - it has been stymied by a max_tokens response allowed of 4096, while the persistent person could get 10000+ out of the prior model, even though OpenAI tried to deny this with training.
And don’t forget paying for “I can’t do that”.
More devday features that don’t work and are degraded and overpromised.