I have access to the gpt-4-1106-preview api, but it’s returning this error when I try to set the max tokens to 6000 (a reasonable amount given the 128K token limit):
max_tokens is too large: 6000. This model supports at most 4096 completion tokens, whereas you provided 6000.
What am I doing wrong?
Never mind. I looked at the docs:
GPT-4 Turbo
New
The latest GPT-4 model with improved instruction following, JSON mode, reproducible outputs, parallel function calling, and more. Returns a maximum of 4,096 output tokens. This preview model is not yet suited for production traffic.
So, I assume that large context for input is so it can read the book, but only 4K tokens available to analyze or summarize it? OK.