I am encountering an issue when using the GPT-4o model compared to GPT-4. We very much want to switch to using GPT-4o because the answers it returns are significantly more accurate and better. The problem is that when we used GPT-4 until now, it did not return answers with markdown in this format some text or ###some text, but GPT-4o does.
We tried adding a section in our LLM prompt to tell GPT-4o that we want the response without markdown, and indeed, we arrived at something that can tell it to do so. However, we noticed that this caused the returned answer to be of lower quality. This could manifest as a less detailed answer and a lack of relevant information to the question.
Therefore, I would appreciate help on how I can tell GPT-4o to return the same answer it would with markdown, only without using markdown.
Itâs important to note that during the build operation, if the build completes or stops, or if the recoater stalls, the recoater automatically moves back to the home position. Also, the recoater blade tabs are delicate. Extra care should be taken to prevent damage to them. If tabs are damaged, the calibrate recoater process will not function properly, resulting in build failures. SOURCE_0 SOURCE_2
I am attaching an example of how I ask him to not use markdown in answer:
Output only plain text. Do not use markdown. Avoid using templates that include â**â or â###â.
Markdown uses multiple asterisks and hashtags. If you are using the API, you can create simple Python code to remove the markdown after GPT4oâs response.
We canât use a 2 prompt pipeline since we need to stream the response, waiting for the entire answer to be streamed and only than clear the markdown syntax char will delay the response from our app.
Well that doesnât make sense. I usually tell it that when it responds, to convert its response to a Microsoft word document. That always works for me.
I get a feeling negative instructions: âdonât do this or donât do thatâ are not working well with gpts. Provide more exact examples of what it can do.
Can I ask why you donât want a markdown formatted output? What formatting (if any) do you want?
Is there some reason you canât use pandoc to just convert the markdown formatted output to either plain text or whatever formatting you actual do want? E.g.,
You are a technical writing expert. Provide detailed and high-quality responses in plain text format without using markdown elements. Do not use **bold**, *italic*, ### headings, or any other markdown-specific formatting in content, in titles or in subtitles. Ensure the responses remain clear and complete. Use the delimiters <<START>> and <<END>> to provide the answer.
Examples of the expected response format:
Example 1:
Question: How does the recoater function in the building operation?
Desired response Desired response (Developer Note: Without **bold**, *italic*, ### headings, or any other markdown-specific formatting):
<<START>>
The recoater moves back to the home position if the build completes or stops, or if the recoater stalls. Extra care should be taken with the recoater blade tabs to avoid damage. If tabs are damaged, the calibration process will fail, leading to build failures.
<<END>>
Example 2:
Question: What are the benefits of bottled water over tap water?
Desired response (Developer Note: Without **bold**, *italic*, ### headings, or any other markdown-specific formatting):
<<START>>
Why Bottled Water Outshines Tap Water: A Clear Choice for Health and Convenience
Health Benefits: Purity in Every Sip
One of the most compelling reasons to choose bottled water over tap water is the rigorous purification process it undergoes. Bottled water companies adhere to strict quality standards to ensure the water is free from contaminants and impurities. This process often includes multiple stages of filtration, ozonation, and ultraviolet light treatment, which remove harmful bacteria, viruses, and chemicals that might be present in tap water. Consequently, bottled water offers a safer and healthier option for consumption, especially for those with compromised immune systems or specific health concerns.
Convenience: Hydration on the Go
Bottled water provides unparalleled convenience, making it an ideal choice for modern, busy lifestyles. Its portability means you can carry it anywhereâwhether youâre at the gym, in the office, or on a road trip. Unlike tap water, which requires you to carry a reusable bottle and frequently refill it, bottled water is ready to drink whenever and wherever you need it. This convenience ensures you stay hydrated throughout the day without the hassle of finding a clean source of tap water.
Taste: Refreshing and Consistent
Another advantage of bottled water is its consistent taste. Tap water can have varying flavors depending on the source and local treatment processes, often leading to a metallic or chlorinated taste. Bottled water, on the other hand, is carefully filtered to ensure a crisp, clean taste with every sip. For those who are particular about the flavor of their water, bottled water provides a reliably refreshing experience, free from the unpleasant tastes that sometimes plague tap water.
Environmental Impact: Recycling and Innovation
While concerns about the environmental impact of bottled water are valid, advancements in packaging technology are addressing these issues. Many bottled water companies are now using recyclable materials and promoting proper disposal practices. Additionally, some brands are exploring biodegradable and plant-based packaging alternatives to reduce their ecological footprint. By choosing brands that prioritize sustainability, consumers can enjoy the benefits of bottled water while minimizing their impact on the environment.
Conclusion: Making the Right Choice
In conclusion, bottled water offers numerous advantages over tap water, from superior health benefits and convenience to consistent taste and evolving environmental practices. While it is essential to be mindful of the ecological impact, the advancements in sustainable packaging make bottled water a viable and beneficial choice. For those seeking purity, convenience, and a refreshing taste, bottled water clearly outshines tap water, making it the preferred option for many consumers.
<<END>>
Below is your task:
<<START>>
Question: {question}
<<END>>
# Example usage
question = "Write an essay about comparison and convincing 'Bottle water is better than tap water.' It should have an eye-catching title and subtitles."
print(get_plain_text_response(question))
Thatâs a lot of tokens to burn to achieve a result which might still include some markdown occasionallyâespecially when you can easily post process the results for a guaranteed solution.
Also, as the OP stated, theyâve seen worse quality results when avoiding markdown.
This will be for two reasons,
The attention mechanism needs to bounce around constantly between the formatting instructions and the content the OP actually wants to generate.
High-quality responses in the pre-training and RLHF data all include markdown formatting, so as generations get progressively longer, when the markdown formatting is missing from earlier in the response the model will move into a lower-quality region of the latent space.
The best path forward is almost always to not fight the model, meet it where it is. Any instructions which arenât geared to producing higher quality content run a significant risk of degrading output quality.
tl;dr: Donât fight the model, let it produce content in the format most associated with high-quality content and handle the formatting yourself in a post-processing step.
You could strip the markdown out in real time using some sort of Regex parsing. Could do this either on the server before sending it back to the client, or on the client. Shouldnât add too much latency.
Then, if youâre saving it to the database, you could use the 2nd prompt to format it better before saving, and then send the final response (from prompt 2) to the user to have it update.
Not perfect, but probably better than displaying markdown.
