GPT-4-32K tokens model for plugins

GPT builders Plugins / Actions builders
, ,
1

GPT-4-32K context limit would be useful for plugins, especially data intensive plugins that give large responses.

Are there any plans to add this?

2

Short answer: no

There’s no current plans for the public rollout of the GPT-4-32k API, it will likely take a long time before there’s computational resources to make the 32k variant available through chatGPT.

1 Like
3

Adding to this. Typically the inference cost of a model scales by roughly the square of the context length. So, GPT-4-32K requires roughly 4x the computing resources of the base GPT-4. As you might well imagine, this is not something OpenAI is likely to just “give away.”

Personally, I think it will require one of,

  1. A new paradigm, some kind of revolutionary new idea for LLMs similar in magnitude to the transformer.
  2. A few generations of hardware advancements, maybe the B100 or whatever comes after it.
  3. A much higher monthly fee.

Possibly some combination of the above.

But, by that point I doubt we will be talking about GPT-4 anymore, though I can imagine perhaps some time in something like 2025 GPT-4-TURBO-64K could become the cheap, default model.

But, no. For the foreseeable future I doubt the 32k context model will become available in Plus, though I would be happy to be wrong.

3 Likes
4

You could also use Claude2. I think its token count is around 100k. It doesn’t score as well as GPT-4 in some measures but not by too much and as well in others. It’s a good looking tool. I’d be using the hell out of them but they won’t give me an API key.