Timeouts on specific prompts

API
1

Hi,

I have an enterprise license key with ChatGPT. What’s happening is and I couldn’t figure out myself, is one specific prompt that I have designed, when I fire it the API returns Timeout Exception. A simple version of the prompt works.

//The one that worked.
What is the value ‘12345678’ in the following PID segment. Answer in following format : [{segment:,field:,component:,subcomponent:,value:}]

//The one that gives timeout
Analyze the PID segments in the following array. Identify the differences. Answer in following format: [{segment:,field:,component:,subcomponent:,value:}]";

Basically, it timesout when I add more PID segments to it in the prompt.
Other settings:
“model”:“gpt-3.5-turbo”,“max_tokens”:512,“temperature”:0.7,“n”:1,“stream”:false,“stop”:null,“top_p”:1,“frequency_penalty”:0,“presence_penalty”:0

I made this in Spring Boot Java.

Any advise?

2

use chatgpt4. you’re using a tokenizer that’s not compatible with 3.5. or write the subscript that handles the arguments different

3

so I change the model=gpt-3.5-turbo to gpt-4?

4

The problem was not with the prompt itself, but the library used to connect to OpenAI APIs.
OkHttpClient defaults timeouts to 10 seconds, but the APIs can respond in 30 seconds depending on how complex the operation is.

This below worked.

client = new OkHttpClient.Builder()
    .connectTimeout(300, TimeUnit.SECONDS)
    .writeTimeout(300, TimeUnit.SECONDS)
    .readTimeout(300, TimeUnit.SECONDS)
    .build();
1 Like
5

Thanks for coming back to let us know.

May this post help someone in the future…