Gpt-3.5-turbo-0125 max 4,096 output tokens

Hello everyone, I have an app that converts a recording into text and then passes it to GPT-3 for processing. To avoid token problems, I used to divide the text into blocks of 4050 characters. The text I need to convert is at most 30,000 characters. In theory, I shouldn’t have any problems with the input. But what about the output? At the moment, I’m using the Assistant API. For completeness, I’ll send you the code of the request I make to the API.

 def format_text(self, text):
        if not text:
            return None
        formatted_text = ""
        #Testing 50000
        max_length = 50000  # Lunghezza massima per blocco di testo
        # Suddividi il testo in blocchi e invia ogni blocco a ChatGPT
        for i in range(0, len(text), max_length):
            block = text[i:i + max_length]
            prompt = self.custom_prompt+ ":\n\n" + block
                self.assistant.add_message_to_thread("user", prompt)
                response = self.assistant.process_message()  # Ottieni la risposta per il blocco corrente
                if response:
                    formatted_text += response + "\n\n" 
            except Exception as e:
                print(f"Errore durante l'elaborazione: {e}")
                logging.error(f"Errore nell'elaborazione della risposta: {e}")

                break  # Interrompi il ciclo in caso di errore

Edit: I tried to send 20000 characters, I had no problems. The issue occurred with the output. It simply lost information that, if limited to 4000 characters, it wouldn’t have lost. It simply summarized my text, losing important information. Do you think there’s a way to avoid it? Or is the best choice to send smaller blocks of text?

1 Like