Gpt-3.5 / 4 model documentation Nov 7 2023 still has inaccuracies about "snapshots"

updated November 7 against all-new documents

While the model documentation has been helpfully clarified that the stable model names are a pointer, an alias, which is an improvement for communication, model numbers and their present condition (and what OpenAI does with models) is still misrepresented.

Model Description Context window Training data
gpt-3.5-turbo Currently points to gpt-3.5-turbo-0613. Will point to gpt-3.5-turbo-1106 starting Dec 11, 2023. See continuous model upgrades. 4,096 tokens Up to Sep 2021
gpt-3.5-turbo-0613 Snapshot of gpt-3.5-turbo from June 13th 2023. Will be deprecated on June 13, 2024. 4,096 tokens Up to Sep 2021
gpt-3.5-turbo-0301 Snapshot of gpt-3.5-turbo from March 1st 2023. Will be deprecated on June 13th 2024. 4,096 tokens Up to Sep 2021
  • The -0613 is NOT a “snapshot from June 13”. It may have been CREATED, that date, but OpenAI has continued to train and alter the model, it’s computing underpinnings, and its cognitive abilities.

  • Likewise, -0301 is where that model was left in June after a fervor of three months of training changes and patches.

The models page says:

We also offer static model versions that developers can continue using for at least three months after an updated model has been introduced.

  • This promise was broken (along with the whole 0301 model being broken for a while).

Much more truthful, and also needed for GPT-4, without also detailing the coarse declines in API programmability, performance quality, and dates of developer applications breaking from changes:

Model Description Context window Training data
gpt-3.5-turbo-0613 gpt-3.5-turbo model created June 13th 2023, supporting function calls. Still under a policy of continued proprietary refinements, last updated November 2023. Will be shut down on June 13, 2024. 4,096 tokens Up to Sep 2021
gpt-3.5-turbo-0301 Introduction of gpt-3.5-turbo model created March 1st 2023, with continued updates through June 2023, and architecture alterations performed October 2023. Will be shut down on June 13th 2024. 4,096 tokens Up to Sep 2021

OpenAI can align itself to be helpful, harmless, and honest.

1 Like

nope, it’s the same model. they really are static. do you have any evidence to the contrary?

1 Like

Things that worked, don’t work.

Ugh, if this forum was hard to search, my propensity for screenshots makes it worse…

First: what model writes the titles for ChatGPT? The same one that has to follow developer’s system instructions?

That weekend was the first big hit where I discover that I can’t write the same way as what I know about the model’s behavior had worked. And here, others (although these anecdotes are certainly not a “daily benchmark thread and perplexity dump”)

This is the kind of playground preset that I could have dropped onto the function calling 3.5 model at introduction, written with absurd metacode that the AI loved, written for the forum. I find then I have to fall back to 0301 for success.

After OpenAI’s snafu-ing 0301 a month ago by filling inputs with extra tokens, it never returned and cannot give those same answers.

in the playground preset are not multi-shot, but the actual AI outputs. I’ll let you progressively erase some of the answers and try it on our handful of models today.

A lot of my chagrin is also at ChatGPT, so it’s hard to separate some of the frustration. The same 3.5 0-shot that took my two functions and gave me back drop-in altered two functions now wraps it in import statements and a class and init and made up variables and main — just because. (yes, I did that to an old chat just a few days ago)

This forum two weeks ago is again “what did they do??”

Thanks for at least hearing me out.

Correction: -0301 wasn’t directly and regularly updated. That one was a snapshot, while from that date, the gpt-3.5-turbo model diverged from it, with the updates of being the live model. That gpt-3.5-turbo was abandoned to nowhere with the switchover at the end of June to the “pointer method” pointed at -0613.

chatgpt does change over time and it’s definitely possible for it to get worse at things you care about. the API models really are intended to be static snapshots. if something changed, it’s a bug.