How to improve streaming’s structure? It’s counting titles as numbers and bulletings, when streaming, it makes all text together with no spaces, but once streaming is completed, that’s when it kinda fixes the structure but still not good enough.
Using a markdown formatter as well
What formatter are you using? what does the raw text look like?
We are using markdown js for formatting text on app. Raw text is what we see when the response is streaming. Had image attached on original post
Very similar to something like this.
do you still have that in a variable? it’s really painful to look at
I wanted to check if your spaces and indents and hashes are set correctly. Ideally, you should be able to paste the content here, for example, and the content should format properly.
There could be multiple issues I could think of off the top of my head:
- your formatter (well ofc it’s markdown, but which markdown formatter/parser) might have an issue/ isn’t built for what you’re trying to do
- you might acidenttally be mangling your spaces somehow when you append your deltas.
Here’s a raw streaming output that parses just fine:
Oy, tumang! To be a real beltalowda, you’ve gotta immerse y’self in the Belt, sasa ke? Get to know the life, the hardships, and the people. Here’s what ya gotta do:
Lang Belta: You gotta learn to speak the Lang Belta, the creole language we Belters use. It’s a mix of English, langiches of Earth and Mars, and many others. It ain’t easy, but it’s part of who we are.
Sosyal Milowda: Spend time with Belters. We’re a community, and we look out for each other. You’ve gotta earn the trust of Beltalowdas, show 'em you’re not just some Earther or Martian playing dress-up.
Tenye Wanya: Get experience working on ships or stations, know how to fix a scrubber or patch up a hull. Oso na techs are key to survival out here.
Kowlting: Understand our culture and our struggle. Belters have always got the short end of the stick, workin’ the worst jobs and getting the least respect. You’ve gotta feel that injustice in your gut.
What could be the top best markdown to use for a proper streaming structure of gpt4 turbo?
marked seems to have a lot of stars, when you google github flavored markdown