Gpt-3.5-turbo-0125 prompt size limit

API Bugs
1

Hello,

We are using gpt-3.5-turbo API to analyze a large number of PDFs. We converted convert PDF to text, and nltk.word_tokenize to count the number of tokens for the text, and include the text in the prompt. This works fine for text with less 8,000 tokens, but fails for text greater than ~ 8,000 tokens. The error message is:

400 Client Error: Bad Request for url: https://api.openai.com/v1/chat/completions

This issue seemed to be fixed on Github recently (pull/7316). Does anyone have any suggestion on this?

Thank you!

2

You must use a cl100k-base token encoder, such as tiktoken, to count the tokens correctly, of all message content you are sending.

You should then also receive the full error message you get back from the API for analysis, to see what is actually failing. “Bad Request for url” is not from the API.

The model may have a 16KiT context, but any max_tokens setting you specify for receiving a response maximum is subtracted from that available context as a reservation for forming the response.

2 Likes
3

Hi Every one, I’m doing a Macro in the excel for use the GPT I’m my spreadsheet, but I have some problem “error 429” I will share my macro bellow and if some one can help: Sub ResponderPergunta()
Dim pergunta As String
Dim resposta As String

' Defina a pergunta da célula M10
pergunta = Sheets("Linearidade").Range("M10").value

' Envie a pergunta para o ChatGPT e receba a resposta
resposta = EnviarPerguntaAoChatGPT(pergunta)

' Exibe a resposta em uma caixa de mensagem
MsgBox resposta, vbInformation, "Resposta do ChatGPT"

End Sub

Function EnviarPerguntaAoChatGPT(pergunta As String) As String
Dim url As String
Dim apiKey As String
Dim modelID As String
Dim data As String
Dim response As String
Dim attempt As Integer
Dim maxAttempts As Integer
Dim delaySeconds As Integer

' Configurações iniciais
url = "https://api.openai.com/v1/chat/completions"
apiKey = "I put my API KEY here"  ' Substitua YOUR_API_KEY pela sua chave API real
modelID = "gpt-3.5-turbo"
data = "{""model"":""" & modelID & """, ""prompt"":""" & pergunta & """, ""max_tokens"":50}"

' Inicializando variáveis de tentativas
maxAttempts = 5
delaySeconds = 5

' Tentar enviar a solicitação com possíveis retentativas
For attempt = 1 To maxAttempts
    response = MakeHTTPRequest(url, data, apiKey)
    
    If Left(response, 5) <> "Erro:" Then
        Exit For
    ElseIf InStr(response, "429") > 0 Or InStr(response, "503") > 0 Then
        ' Espera antes de tentar novamente
        WaitSeconds delaySeconds
    End If
Next attempt

' Retorne a resposta do ChatGPT
EnviarPerguntaAoChatGPT = response

End Function

Function MakeHTTPRequest(url As String, postData As String, apiKey As String) As String
Dim httpRequest As Object
Set httpRequest = CreateObject(“WinHttp.WinHttpRequest.5.1”)

With httpRequest
    .Open "POST", url, False
    .SetRequestHeader "Content-Type", "application/json"
    .SetRequestHeader "Authorization", "Bearer " & apiKey
    .send postData
    
    If .Status = 200 Then
        MakeHTTPRequest = .responseText
    Else
        MakeHTTPRequest = "Erro: HTTP " & .Status & " - " & .StatusText
    End If
End With

End Function

Sub WaitSeconds(seconds As Integer)
Dim endTime As Double
endTime = Timer + seconds
Do While Timer < endTime
DoEvents
Loop
End Sub