GPT-4 8k token limit is gone

API
1

Hi,

Before the keynote yesterday I had access to GPT-4 with an 8K token window just by using the model “gpt-4”. Today, however, it is maxed out at only 2048 tokens.

image

What is happening and why? This breaks my workflow.

Thanks in advance

EDIT: Looks like a bug. Even gpt-3.5-16k only has 2048 tokens available. Someone shipped without first testing the code.

3 Likes
2

is this a visual bug or does it really stop at 2048 ?

1 Like
3

Upon further investigation, yes, it’s only a visual bug.

I fixed it myself for the time being.

Bugfix (click me)

For anyone curious:

image
image1205×792 100 KB

  1. Open up DevTools (F12)
  2. Go into the “Sources” tab and search for the file main.590748c4.js
  3. Ctrl+F (Search) for the exact line: return I[e]
  4. Once localized, place a breakpoint (place a red dot on the left side of the code - clicking)
  5. Move over to the browser window and move the “Maximum length” slider. It should gray the screen instantly and stop at the breakpoint you just set, just like in the image below:
    image
  6. Great! You now have access to the scope variables. Type following into the “Console”: I = E.
  7. If successful, you should see following response:
    image
    If you’re getting an undefined error, the breakpoint didn’t trigger and you don’t have access to the scope and you should start over.
  8. Remove the breakpoint (the red dot, else it will pause again).
  9. Finally, press the “Play/Triangle” icon at the top of the browser or hit F8 to continue.

image
:slight_smile:

I tried to write a script which can be pasted inside the console alone, but the scopes wouldn’t let me (inside anonymous function calls, etc), so it requires more steps. But hey, at least it works!

Note that this change does not persist between tab/browser closes. Do it once and leave the browser open. It’s possible to retain the changes by making a local copy and then letting it load that instead, but let’s not get into that.

Also, some lengths are not really working server-side’ish, like the GPT 128k preview.

EDIT: Not needed anymore. They fixed it. That was fast!

2 Likes
4

Same topic just a scroll or search away:

5

Just limiting it without telling anybody why is not really great.
I get helping novices, but just limiting it for all users, even those who had months of experience with this tool, is just bad.
At least include a toggle somewhere, in the lines of: “I understand that a loop can cost me literal dollars in 128k mode, but I understand and want to continue”.