GPT-4-o returns answers with Markdown

I am encountering an issue when using the GPT-4o model compared to GPT-4. We very much want to switch to using GPT-4o because the answers it returns are significantly more accurate and better. The problem is that when we used GPT-4 until now, it did not return answers with markdown in this format some text or ###some text, but GPT-4o does.

We tried adding a section in our LLM prompt to tell GPT-4o that we want the response without markdown, and indeed, we arrived at something that can tell it to do so. However, we noticed that this caused the returned answer to be of lower quality. This could manifest as a less detailed answer and a lack of relevant information to the question.

Therefore, I would appreciate help on how I can tell GPT-4o to return the same answer it would with markdown, only without using markdown.

It’s important to note that during the build operation, if the build completes or stops, or if the recoater stalls, the recoater automatically moves back to the home position. Also, the recoater blade tabs are delicate. Extra care should be taken to prevent damage to them. If tabs are damaged, the calibrate recoater process will not function properly, resulting in build failures. SOURCE_0 SOURCE_2

I am attaching an example of how I ask him to not use markdown in answer:

  • Output only plain text. Do not use markdown. Avoid using templates that include ‘**’ or ‘###’.

Thanks :slight_smile:

2 Likes

Simple solution.

Use a 2 prompt pipeline.

Prompt 2 simply removes markdown from output of prompt 1.

Markdown uses multiple asterisks and hashtags. If you are using the API, you can create simple Python code to remove the markdown after GPT4o’s response.

2 Likes

Thanks for your response !
We’re using GPT-4O with the OpenAI API for completion.
We’ll test the logit_bias and the instruction you mentioned.

We can’t use a 2 prompt pipeline since we need to stream the response, waiting for the entire answer to be streamed and only than clear the markdown syntax char will delay the response from our app.

Providing 2 or 3 worked examples (n-shot) will also get you the output you want.

1 Like

Tell it to convert its answer to a Microsoft word document.

That was unhelpful in receiving output wihout markdown.

Untitled

1 Like

Well that doesn’t make sense. I usually tell it that when it responds, to convert its response to a Microsoft word document. That always works for me.

I get a feeling negative instructions: “don’t do this or don’t do that” are not working well with gpts. Provide more exact examples of what it can do.

Can I ask why you don’t want a markdown formatted output? What formatting (if any) do you want?

Is there some reason you can’t use pandoc to just convert the markdown formatted output to either plain text or whatever formatting you actual do want? E.g.,

pandoc --from markdown --to plain  --wrap=none

@idan.benmoshe

You may check these chats: HERE and HERE

You are a technical writing expert. Provide detailed and high-quality responses in plain text format without using markdown elements. Do not use **bold**, *italic*, ### headings, or any other markdown-specific formatting in content, in titles or in subtitles. Ensure the responses remain clear and complete. Use the delimiters <<START>> and <<END>> to provide the answer.

Examples of the expected response format:

Example 1:
Question: How does the recoater function in the building operation?
Desired response Desired response (Developer Note: Without **bold**, *italic*, ### headings, or any other markdown-specific formatting):
<<START>>
The recoater moves back to the home position if the build completes or stops, or if the recoater stalls. Extra care should be taken with the recoater blade tabs to avoid damage. If tabs are damaged, the calibration process will fail, leading to build failures.
<<END>>

Example 2:
Question: What are the benefits of bottled water over tap water?
Desired response (Developer Note: Without **bold**, *italic*, ### headings, or any other markdown-specific formatting): 
<<START>>
Why Bottled Water Outshines Tap Water: A Clear Choice for Health and Convenience

Health Benefits: Purity in Every Sip

One of the most compelling reasons to choose bottled water over tap water is the rigorous purification process it undergoes. Bottled water companies adhere to strict quality standards to ensure the water is free from contaminants and impurities. This process often includes multiple stages of filtration, ozonation, and ultraviolet light treatment, which remove harmful bacteria, viruses, and chemicals that might be present in tap water. Consequently, bottled water offers a safer and healthier option for consumption, especially for those with compromised immune systems or specific health concerns.

Convenience: Hydration on the Go

Bottled water provides unparalleled convenience, making it an ideal choice for modern, busy lifestyles. Its portability means you can carry it anywhere—whether you’re at the gym, in the office, or on a road trip. Unlike tap water, which requires you to carry a reusable bottle and frequently refill it, bottled water is ready to drink whenever and wherever you need it. This convenience ensures you stay hydrated throughout the day without the hassle of finding a clean source of tap water.

Taste: Refreshing and Consistent

Another advantage of bottled water is its consistent taste. Tap water can have varying flavors depending on the source and local treatment processes, often leading to a metallic or chlorinated taste. Bottled water, on the other hand, is carefully filtered to ensure a crisp, clean taste with every sip. For those who are particular about the flavor of their water, bottled water provides a reliably refreshing experience, free from the unpleasant tastes that sometimes plague tap water.

Environmental Impact: Recycling and Innovation

While concerns about the environmental impact of bottled water are valid, advancements in packaging technology are addressing these issues. Many bottled water companies are now using recyclable materials and promoting proper disposal practices. Additionally, some brands are exploring biodegradable and plant-based packaging alternatives to reduce their ecological footprint. By choosing brands that prioritize sustainability, consumers can enjoy the benefits of bottled water while minimizing their impact on the environment.

Conclusion: Making the Right Choice

In conclusion, bottled water offers numerous advantages over tap water, from superior health benefits and convenience to consistent taste and evolving environmental practices. While it is essential to be mindful of the ecological impact, the advancements in sustainable packaging make bottled water a viable and beneficial choice. For those seeking purity, convenience, and a refreshing taste, bottled water clearly outshines tap water, making it the preferred option for many consumers.
<<END>>


Below is your task:
<<START>>
Question: {question}
<<END>>


# Example usage
question = "Write an essay about comparison and convincing 'Bottle water is better than tap water.' It should have an eye-catching title and subtitles."
print(get_plain_text_response(question))

That’s a lot of tokens to burn to achieve a result which might still include some markdown occasionally—especially when you can easily post process the results for a guaranteed solution.

Also, as the OP stated, they’ve seen worse quality results when avoiding markdown.

This will be for two reasons,

  1. The attention mechanism needs to bounce around constantly between the formatting instructions and the content the OP actually wants to generate.
  2. High-quality responses in the pre-training and RLHF data all include markdown formatting, so as generations get progressively longer, when the markdown formatting is missing from earlier in the response the model will move into a lower-quality region of the latent space.

The best path forward is almost always to not fight the model, meet it where it is. Any instructions which aren’t geared to producing higher quality content run a significant risk of degrading output quality.

tl;dr: Don’t fight the model, let it produce content in the format most associated with high-quality content and handle the formatting yourself in a post-processing step.

1 Like

You could strip the markdown out in real time using some sort of Regex parsing. Could do this either on the server before sending it back to the client, or on the client. Shouldn’t add too much latency.

Then, if you’re saving it to the database, you could use the 2nd prompt to format it better before saving, and then send the final response (from prompt 2) to the user to have it update.

Not perfect, but probably better than displaying markdown.

2 Likes

Another option: on the client side, use a markdown displayer so that it can just handle the markdown and format it appropriately.

Yes. Yes & Yes!

Anything that deals with static patterns = better handled through programming. Markdown is a perfect base for transforming content

1 Like