I am almost certain that the traditional gpt-4 model works better. I tried both with temperature 0.7
I used the Playground today with Assistants + gpt 4-1106 iirc for the 1st time in 3 or 4 weeks, i noticed that for the couple of questions i asked (coding questions) the answer was very different in helpfulness to the last time i used it. It basically wouldn’t give me a solution, it said consult the coding community (forums etc) lol! Whereas chatgpt-4 via web browser (i have a plus account) was a great help. EDIT: i take this back, i was mistakenly using the wrong assistant i’d created for doing things other than coding, no wonder it didn’t help much.
1 Like
Can you get "'Json Outputs from using GPT 4Model?
Yes, you just need to be extremely specific when asking.
2 Likes
Yeah, I just had this problem a lot personally myself too.
I was puzzled and stunned to find that the gpt-4 turbo’s maximum input+ouput tokens length is not 128,000, but 4089. The following is my finding:
in my usage panel: GPT-4 Turbo $3.95. This mean I was using GPT-4 turbo,
But when I tried to set max_tokens:100,000 The error is raised as follows:
BadRequestError: 400 max_tokens is too large: 100000. This model s
upports at most 4096 completion tokens, whereas you provided 10000
0.
at APIError.generate (D:\diag2024\node_modules\openai\error.js
:43:20)
at OpenAI.makeStatusError (D:\diag2024\node_modules\openai\cor
e.js:251:33)
at OpenAI.makeRequest (D:\diag2024\node_modules\openai\core.js
:290:30)
at process.processTicksAndRejections (node:internal/process/ta
sk_queues:95:5) {
status: 400,
headers: {
‘access-control-allow-origin’: ‘*’,
‘alt-svc’: ‘h3=“:443”; ma=86400’,
‘cf-cache-status’: ‘DYNAMIC’,
‘cf-ray’: ‘85522ca3e8a24a0e-TPE’,
connection: ‘keep-alive’,
‘content-length’: ‘234’,
‘content-type’: ‘application/json’,
date: ‘Wed, 14 Feb 2024 03:20:17 GMT’,
‘openai-model’: ‘gpt-4-1106-preview’,
‘openai-organization’: ‘user-exn1wtriat7vpxb4evwna75k’,
‘openai-processing-ms’: ‘24’,
‘openai-version’: ‘2020-10-01’,
server: ‘cloudflare’,
‘set-cookie’: ‘__cf_bm=3x09VaOgnBTGnBBEOWcMTNcOiVeQgiIS0b3CYge
N.uY-1707880817-1-AdI3+lw5R8lzzgkY5a0MRGtYNHe/TcS/Koo6z+CNysWkaX6i
j5dSmRScPvUKdlq73vhnfCKi894ji/aHObN+rVo=; path=/; expires=Wed, 14-
Feb-24 03:50:17 GMT; domain=.api.openai.com; HttpOnly; Secure; Sam
eSite=None, _cfuvid=hFMbeg8X9WXKZx_R8QtET2S0.LGaidbxDuFjz7Ia78E-17
07880817578-0-604800000; path=/; domain=.api.openai.com; HttpOnly;
Secure; SameSite=None’,
‘strict-transport-security’: ‘max-age=15724800; includeSubDoma
ins’,
‘x-ratelimit-limit-requests’: ‘500’,
‘x-ratelimit-limit-tokens’: ‘500000’,
‘x-ratelimit-remaining-requests’: ‘499’,
‘x-ratelimit-remaining-tokens’: ‘491828’,
‘x-ratelimit-reset-requests’: ‘120ms’,
‘x-ratelimit-reset-tokens’: ‘23m32.121s’,
‘x-request-id’: ‘req_788547e71186bc692a46fb3455b5b355’
},
error: {
message: ‘max_tokens is too large: 100000. This model supports
at most 4096 completion tokens, whereas you provided 100000.’,
type: ‘invalid_request_error’,
param: ‘max_tokens’,
code: null
},
code: null,
param: ‘max_tokens’,
type: ‘invalid_request_error’
}
the api used as follows:
const gptModel=[“gpt-3.5-turbo”,“gpt-4-1106-preview”];
const maxTokens=100000;
openai.chat.completions.create({
model: gptModel[1],
messages: [{“role”: “user”, “content”:prompt}],
max_tokens:maxTokens,
“response_format”:{“type”: “json_object”}
}).then(chatCompletion =>{
resolve(chatCompletion.choices[0].message.content)
})
but it is. the only thing you overlooked that max output length is 4096. the max context window is 128,000 https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo
max_tokens refers to the max number of tokens you want returned; the max of which is 4096 in this case. Or (128000 - input tokens), whichever is lower
1 Like
Thanks. You are right. It’s my mistake.