I’m calling Conversation.AppendSystemMessage and passing the following:
"Generate a detailed SOAP note of the following transcript between a doctor and a patient. However first, here is an example SOAP note for an example transcript. Please follow it’s general format.
I put a long example here in an XML tag named ExampleTranscript
I put an example SOAP note here in an XML tag named ExampleSOAPNote
I put the actual transcript here in an XML tagged named ActualTranscript
"
It worked for a few days and then it started just regurgitated the Example SOAP note verbatim. If I do not provide an example SOAP note the engine creates a SOAP note with such different formatting and verboseness each time.
simple fix: Use placebolders in your examples. something like (DATA_1) and so on. after each format example you tell it to replace the Placeholder DATA_1 with the correct value. Add a sentence like “avoid using DATA_1 as value. Its just an example.” this did the trick for me. i take my replies as json array and it works since months. Even as gpt replies in json anyways, i just had to fix a few patterns like ``` before and after the reply as it sometimes puts it into the codeblock and sometimes not.
Ideally you would identify the text elements within your soap message, then depending on how much precision you want and the complexity of the structure, you would either request the model to produce json response with the fields populated with text you want, or if you want to get to the absolute precision, you would run extractors as separate API requests to get the plain text responses for each field in parallel, and then use classic code to format your soap message.
2 Likes
I kept it simple by creating a GPT to do this. I trained it by giving it several examples of what SOAP notes look like. I also ask it to make sure to reference as many peer-reviewed, online available medical journals and make sure it is 99.9% sure of its response. I also tell it to make sure it asks me any questions if it is unsure of the task it needs to complete. Alll of this is done in the training of the GPT while I am creating it. I can now feed my notes into the GPT for it to generate the SOAP structured note. You can avoid hallucinations by making sure the current context window you are working in is within the token limitations. This way the GPT does not loose the plot and start to repeat useless information. I have tried it to let me know when it is close the token limit in the current context window. Hope this helps.
1 Like