I have been trying gpt-4-1106-preview and gpt-3.5-turbo-1106.
If I understood correctly, these models should now have data until April 2023. However, I have trouble getting more recent information. For example, “Is FTX operational” will claim it is operational on gpt-3.5-turbo-1106. On gpt-4-1106-preview I get correct answer on OpenAI API, but on on Azure OpenAI API. I chose this as an example because it has been widely discussed and widely in the news.
I get that these are not exactly databases or knowledge graphs, but I wonder I have misunderstood something.
My understanding is:
- all 1106 models are trained until april 2023
- models on azure and openai api are the same
I also wonder is there some way to suggest it to consider the latest information, to help it only use up-to-date information. For example, some magical prompt 
Thanks, so gpt-3.5-turbo 1106 is actually sep 2021 cutoff.
The list is missing the new gpt4-1106-preview model, they are already available on both OpenAI API and Azure OpenAI API. Probably because it is a preview model, but that should have more recent data according to devday announcements.
“GPT-4 Turbo is more capable and has knowledge of world events up to April 2023.” New models and developer products announced at DevDay (It says only gpt-4 so it should confirm that 3.5 is only sep 2021)
I still have trouble making GPT-4 Turbo actually use that data after 2021 on Azure. I wonder if it is an Azure issue, model issue, or something else?
ok, it seems Azure issue so not really relevant herre. The deployment may need to be re-created, and the update of existing deployments does not work.
The second points by _l was that only gpt-4 is updated, not gpt-3.5
I opened a support incident with Azure, who claimed that the gpt4-1106-preview just needed to be ‘reminded’ that its training data cutoff is April 2023. I tried that, and it seemed to start producing responses that included some info past December 2021. However the OpenAI version of the gpt4 turbo preview consistently responds with clear knowledge of events up to April 2023. The question I use to determine if I’m dealing with the actual gpt4 turbo 1106 preview is to ask when GPT 4 was released. The OpenAI gpt4-1106-preview consistently responds with March 14th 2023. The Azure gpt4-1106-preview never shows knowledge of this date, regardless of prompting. My suspicion is that they either have a fine tuned version of 0613, or its actually the gpt 3.5 turbo model. No way to know, because Azure support is doubling down on their assertion that its the gpt4-1106-preview. It could be that their content filter is jacking up the response. They don’t offer the ability to turn it off. My intuition from working with gpt4 1106 and 0613 daily is that the Azure 1106 preview is not the same model as the OpenAI version. It sucks because our OpenAI credits expired the day of the new model release, and our Azure credits are all we have for product development, so we either fork over cash for the OpenAI model, or use an inferior model for our business experiments. Not a great way to build the foundation for a successful product.
1 Like
I also have now a ticket open, and I 100% agree with you it is clearly not the same “1106”. I also did lots of calls and spent about ~$500 before noticing this. So now I have thousands of results to compare, and even the file sizes are 20-30% bigger on OpenAI’s side meaning the replies are longer for same prompt and settings. I even use the same class library by Microsoft.
I tried multiple datacenters, updating, deleting and creating.
Maybe the Azure one could also different checkpoint of 1106 or maybe GPT-4-32K. Anyway, at the moment it is best not to try 1106 on Azure, or at least verify it is the “real” 1106 by comparing outputs to questions about events in early 2023.
