Automatic inference of punctuation

Dialog act labelling is a huge problem but at the very least we can add in punctuation (.!?) quickly and easily. This can help for problems such as ASR where we speak without apparent punctuation.

image

2 Likes

Thanks for sharing! This is an interesting use-case.

2 Likes

I am working with dialogue state tracking with GPT-3 and have interesting results. If you are working on something similar, feel free to reach out.

2 Likes

I have actually found that GPT-3 doesn’t really need conversation modeling. It has a pretty solid intuition of conversations, which seems to supersede the need for dialog acts. I’m happy to discuss further if you’re still interested.

1 Like

Yes, you are right. I am working on cases in which the dialogue level annotations are meaningful for example to make externally API calls. In this case, I cannot supersede the need for it.

1 Like

Hi Dave, I have been asked to try and resolve the following. If you have any advice or thoughts, that would be much appreciated. (I’m a newbie on here, so forgive any protocol that I get wrong).

The background is, we have charity workers who are field based and they have to provide feedback in text format (not voice)via laptops and other devices. Understandably, the punctuation is often very poor (or absent!). The challenge is to somehow automate the punctuation, so a huge bit of free text can be successfully broke up into sentences. [The sentences will then be able to go to our pre-existing process where we can generte insights from the sentences]. So basically it’s the auto-punctuation I’m wanting to pursue.

I was able to accomplish this using standard prompt design techniques (providing an instruction and an example), on davinci-instruct-beta.

As you can see, the API successfully adds most of the punctuation that I removed, including the comma after “Hi Dave” and a period at the end of each of the three sentences.

The prompt could be further improved with additional (and higher-quality) examples.

4 Likes

Thanks Joey. So - if I understand correctly, it would be down to us to provide all the examples? I was hoping there might be something that would do “out of the box” punctuation (to a reasonable, albeit not perfect, level). Cheers - Mark

Any usage requires that you provide examples and instructions. Examples are used to show the API what you want it to do.

Got it. I guess thats only to be expected, because it’s so powerful, and used in so many different scenarios (and not just this one [punctuation] use case). Thanks Joey