Understanding which models are available to call with the APIs

Hi all. I’m using the openai.models endpoint to fetch available models and allow the user to make a selection. I see about 38 models, however most of them if I try and select and then use result in errors. I did a loop around available models and tested a conversation API call to see which ones worked, the output (shown at the end of the message) indiciates that it doesn’t seem to be a case of whether the owner_id is openai or system and so on.

Is there any way to determine programatically what models are available to actually use via the APIs? Or is it a case of maintaining your own index of available models?

Here’s the output for reference:

gpt-4.5-preview (system) - ok
gpt-4.5-preview-2025-02-27 (system) - ok
gpt-4o-mini-2024-07-18 (system) - ok
gpt-4o-mini-audio-preview-2024-12-17 (system) - err
dall-e-3 (system) - err
dall-e-2 (system) - err
o3-mini (system) - ok
gpt-4o-audio-preview-2024-10-01 (system) - err
gpt-4o-audio-preview (system) - err
gpt-4o-mini-realtime-preview-2024-12-17 (system) - err
gpt-4o-mini-realtime-preview (system) - err
o1-mini-2024-09-12 (system) - ok
o1-mini (system) - ok
omni-moderation-latest (system) - err
gpt-4o-mini-audio-preview (system) - err
omni-moderation-2024-09-26 (system) - err
whisper-1 (openai-internal) - err
gpt-4o-realtime-preview-2024-10-01 (system) - err
babbage-002 (system) - err
chatgpt-4o-latest (system) - ok
tts-1-hd-1106 (system) - err
text-embedding-3-large (system) - err
gpt-4o-audio-preview-2024-12-17 (system) - err
gpt-4 (openai) - ok
gpt-4o-2024-05-13 (system) - ok
tts-1-hd (system) - err
o1-preview (system) - ok
o1-preview-2024-09-12 (system) - ok
gpt-4o-2024-11-20 (system) - ok
o1-2024-12-17 (system) - ok
gpt-3.5-turbo-instruct-0914 (system) - err
gpt-4o-mini-search-preview (system) - ok
o1 (system) - ok
tts-1-1106 (system) - err
davinci-002 (system) - err
gpt-3.5-turbo-1106 (system) - ok
gpt-4o-search-preview (system) - ok
gpt-4-turbo (system) - ok
gpt-4-0125-preview (system) - ok
gpt-3.5-turbo-instruct (system) - err
gpt-4o-mini-search-preview-2025-03-11 (system) - ok
gpt-3.5-turbo-0125 (system) - ok
gpt-4o-2024-08-06 (system) - ok
gpt-4o-realtime-preview-2024-12-17 (system) - err
gpt-3.5-turbo (openai) - ok
gpt-4-turbo-2024-04-09 (system) - ok
gpt-4o-realtime-preview (system) - err
gpt-3.5-turbo-16k (openai-internal) - ok
gpt-4o (system) - ok
text-embedding-3-small (system) - err
gpt-4-1106-preview (system) - ok
text-embedding-ada-002 (openai-internal) - err
gpt-4-0613 (openai) - ok
o3-mini-2025-01-31 (system) - ok
gpt-4o-mini (system) - ok
gpt-4o-search-preview-2025-03-11 (system) - ok
tts-1 (openai-internal) - err
gpt-4-turbo-preview (system) - ok

At this risk of answering my own question, I notice that there are details like the below - possibly the issue is that all returned models are valid, but only a few valid for completions API? If that’s the case is it possible to programatically determine which models are available for the completions / assistants API?

More and more models are getting deprecated. We used a gpt-4-turbo model with Chat Completions and that one became unusable with Chat Completions the last week this February.

Yeah seems like it’s a nightmare. Have created a repo to track models and metadata, have scaffolded the current list with chatgpt deep research, am now intergrating it into my Terminal AI app and each model I actually test I’ll update the repo with ‘validated’ next to the model. Seems like the only way to manage is to essentially create a DB

Open to any suggestions or contributions if you fancy it, would love to add a python module. Am actually testing the models at the moment based on this

GPT 4 Turbo is still available though:

https://platform.openai.com/docs/models/gpt-4-turbo

(this model is important to one of my workflows too)

this list is an interesting work … could you include if a model is compatible with “assistants API” and with the new “response API” ? Of course these are openai specific but …

The models endpoint could have “features” metadata, such as which endpoint can be used, or specifically, which are “chat” models. But it does not.

Here’s a function that will return the typical text chat models of your account. You can apply the filter criteria to your own code and language.

import asyncio

async def get_chatmodels():
    from openai import AsyncOpenAI as ModelClient
    client = ModelClient()  # uses environment variable API key
    starts_with = ["gpt", "ft:gpt", "o"]
    blacklist = ["instruct", "omni", "realtime", "audio", "search"]
    model_response = await client.models.list()
    model_dict = model_response.model_dump().get('data', [])
    model_list = sorted([model['id'] for model in model_dict])
    filtered_models = [
        model for model in model_list
        if any(model.startswith(prefix) for prefix in starts_with)
        and not any(bl_item in model for bl_item in blacklist)
    ]    
    return filtered_models

async def main():
    try:
        chat_model_list = await get_chatmodels()
        print("--------\n" + "\n".join(chat_model_list))
    except Exception as err:
        print("An error occurred:", err)

if __name__ == "__main__":
    asyncio.run(main())

If the model name ends with two numbers, it can be assumed to be a version snapshot model or a fine tune step model, instead of the duplicate alias pointing to the current recommended model of that type.

2 Likes

working on it now… looks a bit like this:

    models:
      - id: o3-mini-2025-01-31
        name: o3-mini
        documentation_url: https://platform.openai.com/docs/models/o3-mini
        description_short: Fast, flexible, intelligent reasoning model
        description: o3-mini is our newest small reasoning model, providing high intelligence at the same cost and latency targets of o1-mini. o3-mini supports key developer features, like Structured Outputs, function calling, and Batch API.
        status: preview
        knowledge_cutoff: October 2023
        context_window: 131072
        max_output_tokens: 8192
        pricing:
          input_per_million: 0.15
          output_per_million: 0.6
        modalities:
          text: true
          image: false
          audio: false
        endpoints:
          assistants: true
          batch: false
          completions_legacy: false
          embeddings: false
          fine_tuning: false
          image_generation: false
          moderation: false
          realtime: false
          responses: true
          speech_generation: false
          transcription: false
          translation: false

Just doing some testing on the nodejs version of the code

1 Like

Not the Azure turbo version we were using.

1 Like

That’s annoying. Good model if pricey.

1 Like

FYI @tilleul the API capabilities have been added now, at least for all of the models you can use on the ChatGPT website (which is what I’ve started with):

Essentially the YAML looks like:

# ...
        endpoints:
          assistants: true
          batch: false
          completions_legacy: false
          embeddings: false
          fine_tuning: true
# ...

I have a nodejs module deployed so you can access all of these modules in node, am working on python and will extend the models as I need them

1 Like

My best effort work in progress is here:

I’m using this on my hobby project: terminal-ai

So far I’m validating each field by hand as even Deep Research is not getting them right (i.e. each parameter is checked by me, pricing etc)

1 Like

I did something like this, which extended the models endpoint return with more metadata. The source was JSON, and beyond the obvious dozen fields that could or could not apply (like new fine-tuning price upon a fine-tune model, or not maximum output on embeddings), I had created feature keywords that were just in a list of features per model (like “json_schema”).

Ultimately the maintenance and forum posting only serving others was abandoned because application needs model needs features needs parameters, needs evaluation, and this would only be useful to a “playground”.

Yes I think a difficult one to maintain, essentially creating both a taxonomy and database of a very rapidly changing domain, I need something ‘good enough’ for a bit of hobby work but would be surprised if it’d go beyond that

great work ! thanks ! now the hard part begins: keep this list up to date :wink: