Hi all. I’m using the openai.models endpoint to fetch available models and allow the user to make a selection. I see about 38 models, however most of them if I try and select and then use result in errors. I did a loop around available models and tested a conversation API call to see which ones worked, the output (shown at the end of the message) indiciates that it doesn’t seem to be a case of whether the owner_id is openai or system and so on.
Is there any way to determine programatically what models are available to actually use via the APIs? Or is it a case of maintaining your own index of available models?
Here’s the output for reference:
gpt-4.5-preview (system) - ok
gpt-4.5-preview-2025-02-27 (system) - ok
gpt-4o-mini-2024-07-18 (system) - ok
gpt-4o-mini-audio-preview-2024-12-17 (system) - err
dall-e-3 (system) - err
dall-e-2 (system) - err
o3-mini (system) - ok
gpt-4o-audio-preview-2024-10-01 (system) - err
gpt-4o-audio-preview (system) - err
gpt-4o-mini-realtime-preview-2024-12-17 (system) - err
gpt-4o-mini-realtime-preview (system) - err
o1-mini-2024-09-12 (system) - ok
o1-mini (system) - ok
omni-moderation-latest (system) - err
gpt-4o-mini-audio-preview (system) - err
omni-moderation-2024-09-26 (system) - err
whisper-1 (openai-internal) - err
gpt-4o-realtime-preview-2024-10-01 (system) - err
babbage-002 (system) - err
chatgpt-4o-latest (system) - ok
tts-1-hd-1106 (system) - err
text-embedding-3-large (system) - err
gpt-4o-audio-preview-2024-12-17 (system) - err
gpt-4 (openai) - ok
gpt-4o-2024-05-13 (system) - ok
tts-1-hd (system) - err
o1-preview (system) - ok
o1-preview-2024-09-12 (system) - ok
gpt-4o-2024-11-20 (system) - ok
o1-2024-12-17 (system) - ok
gpt-3.5-turbo-instruct-0914 (system) - err
gpt-4o-mini-search-preview (system) - ok
o1 (system) - ok
tts-1-1106 (system) - err
davinci-002 (system) - err
gpt-3.5-turbo-1106 (system) - ok
gpt-4o-search-preview (system) - ok
gpt-4-turbo (system) - ok
gpt-4-0125-preview (system) - ok
gpt-3.5-turbo-instruct (system) - err
gpt-4o-mini-search-preview-2025-03-11 (system) - ok
gpt-3.5-turbo-0125 (system) - ok
gpt-4o-2024-08-06 (system) - ok
gpt-4o-realtime-preview-2024-12-17 (system) - err
gpt-3.5-turbo (openai) - ok
gpt-4-turbo-2024-04-09 (system) - ok
gpt-4o-realtime-preview (system) - err
gpt-3.5-turbo-16k (openai-internal) - ok
gpt-4o (system) - ok
text-embedding-3-small (system) - err
gpt-4-1106-preview (system) - ok
text-embedding-ada-002 (openai-internal) - err
gpt-4-0613 (openai) - ok
o3-mini-2025-01-31 (system) - ok
gpt-4o-mini (system) - ok
gpt-4o-search-preview-2025-03-11 (system) - ok
tts-1 (openai-internal) - err
gpt-4-turbo-preview (system) - ok
At this risk of answering my own question, I notice that there are details like the below - possibly the issue is that all returned models are valid, but only a few valid for completions API? If that’s the case is it possible to programatically determine which models are available for the completions / assistants API?
More and more models are getting deprecated. We used a gpt-4-turbo model with Chat Completions and that one became unusable with Chat Completions the last week this February.
Yeah seems like it’s a nightmare. Have created a repo to track models and metadata, have scaffolded the current list with chatgpt deep research, am now intergrating it into my Terminal AI app and each model I actually test I’ll update the repo with ‘validated’ next to the model. Seems like the only way to manage is to essentially create a DB
Open to any suggestions or contributions if you fancy it, would love to add a python module. Am actually testing the models at the moment based on this
this list is an interesting work … could you include if a model is compatible with “assistants API” and with the new “response API” ? Of course these are openai specific but …
The models endpoint could have “features” metadata, such as which endpoint can be used, or specifically, which are “chat” models. But it does not.
Here’s a function that will return the typical text chat models of your account. You can apply the filter criteria to your own code and language.
import asyncio
async def get_chatmodels():
from openai import AsyncOpenAI as ModelClient
client = ModelClient() # uses environment variable API key
starts_with = ["gpt", "ft:gpt", "o"]
blacklist = ["instruct", "omni", "realtime", "audio", "search"]
model_response = await client.models.list()
model_dict = model_response.model_dump().get('data', [])
model_list = sorted([model['id'] for model in model_dict])
filtered_models = [
model for model in model_list
if any(model.startswith(prefix) for prefix in starts_with)
and not any(bl_item in model for bl_item in blacklist)
]
return filtered_models
async def main():
try:
chat_model_list = await get_chatmodels()
print("--------\n" + "\n".join(chat_model_list))
except Exception as err:
print("An error occurred:", err)
if __name__ == "__main__":
asyncio.run(main())
If the model name ends with two numbers, it can be assumed to be a version snapshot model or a fine tune step model, instead of the duplicate alias pointing to the current recommended model of that type.
FYI @tilleul the API capabilities have been added now, at least for all of the models you can use on the ChatGPT website (which is what I’ve started with):
I did something like this, which extended the models endpoint return with more metadata. The source was JSON, and beyond the obvious dozen fields that could or could not apply (like new fine-tuning price upon a fine-tune model, or not maximum output on embeddings), I had created feature keywords that were just in a list of features per model (like “json_schema”).
Ultimately the maintenance and forum posting only serving others was abandoned because application needs model needs features needs parameters, needs evaluation, and this would only be useful to a “playground”.
Yes I think a difficult one to maintain, essentially creating both a taxonomy and database of a very rapidly changing domain, I need something ‘good enough’ for a bit of hobby work but would be surprised if it’d go beyond that
