GPT-4o producing bad linebreaks and invalid JSON

Since the spring update and switching from GPT-4 Turbo to GPT-4o, I’ve noticed that (sometimes, not always) GPT-4o will return invalid JSON or line breaks. This occurs mostly when asking it to produce code.

I would show my initial prompt, but that is confidential- however, when asking it to provide code to fix an issue with a javascript even not firing properly, it gives me this:

An example bad response (which is a JSON structure) would be:

---- GPT-4o RESPONSE ----
{
“thoughts”: “To address the issue where pressing the ‘Escape’ key does not trigger the onCancel function, we need to ensure that the event listener is correctly scoped to the dialog component and that it is properly removed when the dialog is closed. We will also verify that the onCancel function is correctly passed and bound.”,
“action”: “answer”,
“answer”: “To ensure that pressing the ‘Escape’ key triggers the onCancel function only when the dialog is open, you can modify your code as follows:\n\njsx\nimport * as React from \"react\";\nimport Button from \"@mui/material/Button\";\nimport Dialog from \"@mui/material/Dialog\";\nimport DialogActions from \"@mui/material/DialogActions\";\nimport DialogContent from \"@mui/material/DialogContent\";\nimport DialogContentText from \"@mui/material/DialogContentText\";\nimport DialogTitle from \"@mui/material/DialogTitle\";\nimport Paper from \"@mui/material/Paper\";\nimport Draggable from \"react-draggable\";\n\nfunction PaperComponent(props) {\n return (\n <Draggable\n handle=\"#draggable-dialog-title\"\n cancel={'[class*=\"MuiDialogContent-root\"]'}\n >\n <Paper {...props} />\n </Draggable>\n );\n}\n\nexport default function DraggableDialog({\n open,\n confirmationButtonText,\n cancellationButtonText,\n title,\n content,\n onConfirm,\n onCancel,\n}) {\n React.useEffect(() => {\n const handleKeyDown = (event) => {\n if (event.key === 'Escape') {\n onCancel();\n }\n };\n if (open) {\n document.addEventListener('keydown', handleKeyDown);\n } else {\n document.removeEventListener('keydown', handleKeyDown);\n }\n return () => {\n document.removeEventListener('keydown', handleKeyDown);\n };\n }, [open, onCancel]);\n\n return (\n <React.Fragment>\n <Dialog\n open={open}\n PaperComponent={PaperComponent}\n aria-labelledby=\"draggable-dialog-title\"\n >\n <DialogTitle style={{ cursor: \"move\" }} id=\"draggable-dialog-title\">\n {title}\n </DialogTitle>\n <DialogContent>\n <DialogContentText>{content}</DialogContentText>\n </DialogContent>\nn <DialogActions>\nn {onCancel && cancellationButtonText && (\nn <Button autoFocus onClick={onCancel}>\nn {cancellationButtonText}\nn </Button>\nn )}\nn <Button onClick={onConfirm}>{confirmationButtonText}</Button>\nn </DialogActions>\nn </Dialog>\nn </React.Fragment>\nn );\nn}\nn\nn### Explanation:\nn1. Scoped Event Listener: The React.useEffect hook now adds the keydown event listener only when the dialog is open and removes it when the dialog is closed.\nn2. Binding onCancel: Ensure that onCancel is correctly passed as a prop and bound to the cancel button.\nnThis should ensure that onCancel fires correctly when you hit cancel and that the keypress event listener only affects the relevant component.”
}
---- END RESPONSE ----

You can see that at the end it starts screwing up the newlines towards the end, which then has a cascading effect and really kills the rest of the message.

1 Like

This has happened to me several times. I found out that if I keep conversing and changing code and implementing GPT-4o on the SAME chat, it will be become sluggish and it will start giving bad linebreaks and some garbage JSON.

My solution: Keep the code iterations in seperate chats and you will no longer see those bad linebreaks and/or invalid JSON.

Perhaps there is a maximum tokenization + memory limit on the code side of GPT. Not sure as I have not found any documentation on this.

2 Likes

I had that experience as well.

I am now testing with the JSON response format, as mentioned in the API (https://platform.openai.com/docs/api-reference/chat/create#chat-create-response_format).

Have you also tried that?

This has happened to me several times. I found out that if I keep conversing and changing code and implementing GPT-4o on the SAME chat, it will be become sluggish and it will start giving bad linebreaks and some garbage JSON.

It will happen with me on a first message, depending on the code. If it’s JSX (which has a bunch of javascript and markdown) it happens a lot more frequently.

Keep the code iterations in seperate chats and you will no longer see those bad linebreaks and/or invalid JSON.

Unfortunately, I can’t do this because of the nature of the application- for the moment I am going to switch back to GPT-4 Turbo, which did not have this issue.

I am now testing with the JSON response format, as mentioned in the API…

I wish it were that simple- I have been using JSON response format since GPT-4 Turbo, as all of the responses go into a JSON structure in my application, so unfortunately that isn’t the issue.

So is there anyone from OpenAI looking at this stuff?

I’ve had to go back to using GPT-4 Turbo because the results from GPT-4o are so bad / unreliable.

Maybe that’s what they want, though?? It’s more expensive, that’s for sure. It’s also slower, but at least it gives me good results.

I’ve taken to using GPT-4o only for minor stuff- can’t rely on it at all.

1 Like

I’m sure that OpenAI would prefer to have you use GPT-4o: GPT-4 pricing, GPT-3.5 expenses.

However one can see the flaws in this plan when you explore the true capabilities of the model and try to build with it.

1 Like

Then they’d better get GPT-4o into shape :slight_smile:

Right now it’s unusable for any coding tasks, due to the random way it stops outputting proper line breaks. (\n starts to become n everywhere at a certain point)

Instead of asking the model to encode the code in a JSON field, can you instead ask it to

Answer with a markdown block containing the code, and then another block containing JSON matching this schema:

Yes, the magic black box might be able to return code as an escaped JSON string, but there’s a lot more code on the internet/training-corpus than escaped JSON strings. Why make the magic black box work harder if you can make it work less?

This actually isn’t limited to code, now I am seeing it happen on any long-ish response from the LLM.

At some point (even just generating raw markdown) GPT-4o starts outputting bad linebreaks.

can you paste the raw response in code blocks into the forum (using backtics)? The formatting at the moment is unclear.

We actually wrote a json parser (in both TS and Python) that can handle new lines or missing quotes even so I could definitely test it out for you and see if it’s something you’re interested in. Works with streaming as well.

To be clear, it’s not a JSON issue- it’s a “GPT-4o goes off the rails” issue.

Here’s an example markdown-only response that it gave me where it just went off on a \ tangent at the end:

# Refactoring Instructions for `Conversations.jsx`

## 1. **Extract API Hooks**

### Files to Create:
- `src/hooks/useConversationsApi.js`
- `src/hooks/useDocumentsApi.js`

### Steps:
1. **Create `useConversationsApi` Hook**:
   - Move the `useConversationsApi` import and related code to a new file named `useConversationsApi.js`.
   - Export the hook from this file.
2. **Create `useDocumentsApi` Hook**:
   - Move the `useDocumentsApi` import and related code to a new file named `useDocumentsApi.js`.
   - Export the hook from this file.
3. **Update Imports**:
   - Update the imports in `Conversations.jsx` to use these new hooks from their respective files.

## 2. **Extract WebSocket Hook**

### File to Create:
- `src/hooks/useWebsockets.js`

### Steps:
1. **Move WebSocket Logic**:
   - Move the `useWebsockets` import and related code to a new file named `useWebsockets.js`.
   - Export the hook from this file.
2. **Update Imports**:
   - Update the imports in `Conversations.jsx` to use this new hook from its respective file.

## 3. **Extract Context Provider**

### File to Create:
- `src/providers/ConversationProvider.js`

### Steps:
1. **Move Conversation Context**:
   - Move the `ConversationContext` import and related code to a new file named `ConversationProvider.js`.
   - Export the context and provider from this file.
2. **Update Imports**:
   - Update the imports in `Conversations.jsx` to use this new context provider from its respective file.

## 4. **Extract Utility Functions**

### File to Create:
- `src/utils/conversationUtils.js`

### Steps:
1. **Move Utility Functions**:
   - Identify utility functions such as `useThrottledCallback`, `handleTextareaFocus`, etc., and move them to a new file named `conversationUtils.js`.
   - Export these functions from this file.
2. **Update Imports**:
   - Update the imports in `Conversations.jsx` to use these utilities from their respective files.

## 5. **Extract Components**

### Files to Create or Update:
- Existing components like `FileDisplay`, `FileSelectionDialog`, and others should be moved/created under a directory like `/components/conversation/`
- Ensure each component has its own file, e.g., `/components/conversation/FileDisplay.jsx`, `/components/conversation/FileSelectionDialog.jsx`, etc.

### Steps:
1. **Move Components**: 																			 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 \-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\-
\- Move existing components such as FileDisplay, FileSelectionDialog, MessageList, DrawerComponent, etc., into their own files within a components/conversation directory.- Ensure each component is properly exported from its own file.- Update Imports:- Update the imports in Conversations.jsx to use these components from their respective files.- Example: import FileDisplay from './components/conversation/FileDisplay';6.Extract Styles (Optional)Files to Create:- src/styles/conversationStyles.jsSteps:1.Move Styles:- If there are inline styles or style-related logic, consider moving them into a separate conversationStyles.js file.- Export these styles as needed.Update Imports:- Update Conversations.jsx to use these styles from conversationStyles.js if applicable.# Final NoteAfter completing these steps, your Conversations.jsx should be significantly cleaner and more modular, making it easier to maintain and understand.

Here is another example of GPT-4o losing its mind when outputting markdown (this was inside of a JSON segment, which was well-formatted and parsable- but obviously the markdown it put in there was not correct):

---- Everything between this and the last ---- is the output it gave me, but can’t be formatted correctly here because the response contains tripple backticks that close my code block -----

I found several code snippets related to voice activation in the repository you provided. Here are the relevant parts:

1. **Initialization of Wake Word**:
```python
self.wake_word = WakeWord(
    wake_word_model_paths=[self.voice_configuration.wake_word_model_path]
)

Source

  1. Starting the Wake Word Detection Thread:
# Start the thread that looks for wake words in the audio_queue
self.look_for_wake_words()

Source

  1. Wake Word Detection Logic:
import time
import datetime
import logging
import os
import threading
from some_audio_library import play_wav_file  # Placeholder for actual import

def look_for_wake_words(self):
    # Set the last activation time to cooldown seconds ago so that we can activate immediately
    last_activation = time.time() - self.voice_configuration.activation_cooldown
    
    # Start listening for wake words
    logging.info("\n\n--- Listening for wake words...\n")
    
    while True:
        # Pull a frame from the queue fed by the mic thread
        frame = self.audio_queue.get(block=True)
        
        # Predict whether a wake word is in this frame
        predictions = self.wake_word.get_wake_word_predictions(frame)
        
        # Get the highest ranked prediction (I only care about the best one)
        prediction = self.wake_word.get_highest_ranked_prediction(
            predictions, [self.voice_configuration.wake_word_model_path]
        )
        
        # Does this prediction meet our threshold, and has enough time passed since the last activation?
        if (
            prediction is not None
            and prediction["prediction"][prediction["wake_word_model"]]
            > self.voice_configuration.model_activation_threshold
            and (time.time() - last_activation)
            >= self.voice_configuration.activation_cooldown
        ):
            detect_time = datetime.datetime.now().strftime("%Y_%m_%d_%H_%M_%S")
            
            logging.info(
                f"Detected activation from '{prediction['wake_word_model']}' model at time {detect_time}!"
            )
            
            # Alert the user we've detected an activation
            play_wav_file(
                os.path.join(
                    os.path.dirname(__file__), "voice/audio", "activation.wav"
                ),
                self.stop_event,
            )
            
            # Detected an activation! Let's process it on a different thread after stopping any other activations
            if self.activation_thread is not None:
                self.stop_event.set()
                self.audio_transcriber.stop_transcribing()
                self.activation_thread.join()
                self.activation_thread = None
                self.stop_event.clear()
            
            self.activation_thread = threading.Thread(
                target=self.process_activation, args=(prediction,)
            )
n           self.activation_thread.start()
n           last_activation = time.time()
n           logging.info("--- Continuing to listen for wake words...")
n```
n[Source](https://github.com/aronweiler/assistant/blob/a8abd34c6973c21bc248f4782f1428a810daf899/src/ai/voice_ai.py#L177-L232)
n4. **Processing Activation**:
n```python
def process_activation(self, prediction):
wake_model = prediction["wake_word_model"]nn# Mute any audio playing if configured to do so.
sound.mute() if self.voice_configuration.mute_while_listening else None
tanscription_start_time = time.time()
slef.audio_transcriber.transcribe_until_silence(RATE, SILENCE_LIMIT_IN_SECONDS)
tanscribed_audio = slef.audio_transcriber.get_transcription()
tanscription_end_time = time.time()
n```
n[Source](https://github.com/aronweiler/assistant/blob/a8abd34c6973c21bc248f4782f1428a810daf899/src/ai/voice_ai.py#L234-L245)
n5. **Voice Configuration Parameters**:
n```json
{
n  "activation_cooldown": 2,
n  "model_activation_threshold": 0.5,
n  "wake_word_model_path": "src/ai/voice/models/hey_jarvis_v0.1.onnx"
n}
n```
n[Source](https://github.com/aronweiler/assistant/blob/a8abd34c6973c21bc248f4782f1428a810daf899/configurations/app_configs/voice.json)
nThese snippets should give you a comprehensive overview of how voice activation is implemented in this repository.

----- End response -----

You can see the linebreaks going berserk at the end.