Hello! We are using the GPT-4 API to write and explain snippets of code. We have it respond using Markdown, which includes text and code blocks (denoted with triple backticks)
Occasionally, GPT-4 returns HTML entities within code blocks instead of the expected characters, for example: <Stack spacing="lg"> instead of <Stack spacing="lg">
Has anyone else encountered this? These are easy enough to decode on the frontend, but would be nice to avoid entirely.
What are you settings? Are you using any penalties? High temp?
It would be tedious but you could try and use logit bias (< is mapped to 33580) to prevent the model drifting to the entities. May suffice with just the < tag?
You could also give some few shot examples to push it away from using those entities (probably better)
But honestly I would think if the quality of content is the same then just re-mapping the characters would be an easy and harmless path.
It would be interesting to do a side-by-side comparison and see what is triggering the change.
When using GPT-4, and accuracy rather than cost or time is paramount, introspection is the most powerful tool at your disposal, asking the model to look at it’s own work with a view to producing a perfect response will usually correct such issues.
Welcome to the Openai @elliotfklein
In the prompt where you ask the model to give markdown, you can ask for “valid markdown”.
Does this happen wrt a specific piece of code(every time) or is it happening randomly?
Sharing any additional hyper params that you’re using would be helpful.
From my experience the following system prompt will probably stop this:
“You are an expert software developer and web designer. When you include HTML in your responses you will format it nicely and indented”
In my experience it’s smart enough do to that, and I’ve only ever even used GPT-3.5-turbo which is great for fast cheap answers to coding questions.
Thanks everyone! Ended up just re-mapping the entities on the frontend