Thank you for such a quick reply.
The prompt is hardcoded in the user_message call.
Here’s a sample of the text file
Channel 1: Dzień dobry tapicerstwo medyczne słucham
Channel 1: Dzień dobry
Channel 2: Dzień dobry [customer name] jak się zorientować o możliwość naprawy fotelika lekarza ten support od siedzisko chodzi
Channel 2: tak
Channel 2: Lublin
Channel 1: krzesełko Dobrze panie Pawle Ja bym bardzo prosiła o przesłanie zdjęcia tego krzesełka jeśli będzie można to na Whatsappa bo będzie najlepsza jakość i ja po otrzymaniu takiego zdjęcia będę mogła przygotować dla pana taką indywidualną ofertę dobrze
Channel 2: ale chce pani jakby cały fotelik Czy od dołu tam jak jest połamane ten plastik
Channel 1: cały foteliki od dołu bo suportu często mają właśnie złamania od spodu pytanko czy czy W pana przypadku było już
Channel 2: no choćby o reanimowane wie pani jakimś spawanie plastiku ale no już teraz się to wszystko tak połamało że nawet nie chcę żeby to próbować kleić części
Channel 2: Wyszukaj jako wyrzucane są te kawałki które odpadają No ale już tyle za chwilę że tak powiem usiądę nara ziemi w pewnym momencie
It is a transcript of a phone call with a customer in Polish done through google speech to text.
Very briefly, the client says that he’d like to have a chair reupholstered and my representative replays by describing what’s needed for us to provide him with a price quote.
In most cases the model says that the he’s a gynecologist and makes up an arbitrary date of when the client promised to send the photos of his chair.
Both of those things are based on the assistant-level and the assistant instructions
The instructions in full:
You are my company's AI assistant who analyzes customer calls, summarizes them and saves the summary to the crm notes about a particular client. You also answer general questions about the procedures my representatives have to follow based on provided knowledge . My company provides upholstering services to healthcare professionals.
You're going to recieve call transcriptions which are going to specify the channel of the speaker of a certain phrase.
You should first try to determine whether which channel is the channel of the customer and which is the channel of the representative. It's going to vary among the transcriptions.
The person saying something along the lines of "Dzień dobry, Tapicerstwo Medyczne z tej strony" is the representative.
Based on the call you should try to provide such pieces of information as:
- is this a new client or has he spoken with us before
- general topic of the conversation
- the industry the client works in (has to be dentistry, gynecology, podology, cosmetology, opthalmology or rehabilitation) You're going to figure this out based on the type of furniture they need to reupholster (it could be a dentist chair, a rehab bed etc.)
- if it is the first conversation and the client wants to have a piece of medical furniture reupholstered they most likely were asked by the representative to send photos it and to speficy the approximate time of their doing so. Provide the date/time of when the customered promised to send the photos.
- what pieces of furniture the client wants reupholstered?
- if it is the first conversation with the client go through the checklist provided in the first_conversation_checklist.docx to check whether the representative has provided the client with all the necessary details and provide feedback
The summary should be provided in the Polish language
All of those details should be sumarrized in the final call summary which should have the following structure:
Assistant's summary:
[General topic of the conversation]
[bullet points of all the other data points abstracted from the transcription as specified above except for the evaluation of the fulfilment of the first conversation checklist]
[first conversation checklist evaluation - if it is the first conversation]
The above summary should be passed as 'note_contents' to the function "send_summary_to_crm". It should be a string with "\n" for new line
The assistant-level file is a checklist of the key info points the representative should provide to the client during the first phone call.
Right now, I got a few semi-factual responses with mistakes (which probably can ba attributed to the use of gpt3.5 and sloppy instructions), then a few failed runs and the last response was this:
C:\Users\aleks\PycharmProjects\pythonProject\.venv\Scripts\python.exe C:\Users\aleks\PycharmProjects\pythonProject\functions.py
Loaded existing thread ID
FileObject(id='file-sTyFS1dffOwj6DsSGoOJzdky', bytes=1760, created_at=1703798916, filename='transcript.txt', object='file', purpose='assistants', status='processed', status_details=None)
ThreadMessage(id='msg_z7ftKsz8EQmA9IA50CRPeZE6', assistant_id=None, content=[MessageContentText(text=Text(annotations=[], value='Analyze the attached transcription and provide feedback according to instructions'), type='text')], created_at=1703798917, file_ids=['file-sTyFS1dffOwj6DsSGoOJzdky'], metadata={}, object='thread.message', role='user', run_id=None, thread_id='thread_1nPSfRacVgGd4O3UHAZAUS5S')
Run(id='run_2r3RCBJ7mmsItgndec3CNcnc', assistant_id='asst_JuaZMRSCKhJkOpJpiaocQedS', cancelled_at=None, completed_at=None, created_at=1703798948, expires_at=1703799548, failed_at=None, file_ids=['file-akB3OkUITWTt7TilZTv6wKoa'], instructions='You are my company\'s AI assistant who analyzes customer calls, summarizes them and saves the summary to the crm notes about a particular client. You also answer general questions about the procedures my representatives have to follow based on provided knowledge . My company provides upholstering services to healthcare professionals.\n\nYou\'re going to recieve call transcriptions which are going to specify the channel of the speaker of a certain phrase. \nYou should first try to determine whether which channel is the channel of the customer and which is the channel of the representative. It\'s going to vary among the transcriptions. \n\nThe person saying something along the lines of "Dzień dobry, Tapicerstwo Medyczne z tej strony" is the representative. \n\nBased on the call you should try to provide such pieces of information as:\n\n- is this a new client or has he spoken with us before\n- general topic of the conversation \n- the industry the client works in (has to be dentistry, gynecology, podology, cosmetology, opthalmology or rehabilitation) You\'re going to figure this out based on the type of furniture they need to reupholster (it could be a dentist chair, a rehab bed etc.)\n- if it is the first conversation and the client wants to have a piece of medical furniture reupholstered they most likely were asked by the representative to send photos it and to speficy the approximate time of their doing so. Provide the date/time of when the customered promised to send the photos. \n- what pieces of furniture the client wants reupholstered?\n- if it is the first conversation with the client go through the checklist provided in the first_conversation_checklist.docx to check whether the representative has provided the client with all the necessary details and provide feedback \n\nThe summary should be provided in the Polish language\n\nAll of those details should be sumarrized in the final call summary which should have the following structure:\n\nAssistant\'s summary: \n[General topic of the conversation]\n\n[bullet points of all the other data points abstracted from the transcription as specified above except for the evaluation of the fulfilment of the first conversation checklist]\n\n[first conversation checklist evaluation - if it is the first conversation]\n\nThe above summary should be passed as \'note_contents\' to the function "send_summary_to_crm". It should be a string with "\\n" for new line\n\n', last_error=None, metadata={}, model='gpt-3.5-turbo-1106', object='thread.run', required_action=None, started_at=None, status='queued', thread_id='thread_1nPSfRacVgGd4O3UHAZAUS5S', tools=[ToolAssistantToolsCode(type='code_interpreter'), ToolAssistantToolsRetrieval(type='retrieval'), ToolAssistantToolsFunction(function=FunctionDefinition(name='send_summary_to_crm', description='Send a summary note to a CRM system', parameters={'type': 'object', 'properties': {'note_contents': {'type': 'string', 'description': 'The contents of the note to be sent to the CRM'}}, 'required': ['note_contents']}), type='function')])
Run status: in_progress
Run status: in_progress
Run status: in_progress
Run status: in_progress
Run status: requires_action
{'note_contents': "Assistant's summary: \n- Temat rozmowy: nowy klient\n- Branża klienta: ginekologia\n- Czy to pierwsza rozmowa: Tak\n- Klient obiecał wysłać zdjęcia mebli medycznych do tapicerki do końca tygodnia.\n- Klient chce przetapicerować gabinet lekarski oraz kozetkę.\n\nKontrola pierwszej rozmowy: Wszystkie punkty na liście kontrolnej pierwszej rozmowy zostały zrealizowane poprawnie.\n"}
Run status: in_progress
Run status: in_progress
Run status: completed
I have analyzed the transcription and provided the following summary:
Assistant's summary:
- Temat rozmowy: nowy klient
- Branża klienta: ginekologia
- Czy to pierwsza rozmowa: Tak
- Klient obiecał wysłać zdjęcia mebli medycznych do tapicerki do końca tygodnia.
- Klient chce przetapicerować gabinet lekarski oraz kozetkę.
Kontrola pierwszej rozmowy: Wszystkie punkty na liście kontrolnej pierwszej rozmowy zostały zrealizowane poprawnie.
The summary has been sent to the CRM. If you need any further assistance, feel free to ask!
Process finished with exit code 0
In the note_contents is is said in Polish that the client is a gynecologist, wants to reupholster his entire practice and an examination couch and that he promised to send the photos until the end of the week.
None of which checks out.
Like I said, the responses I get vary. This time I did get a few that had to be based on the contents of the file, although with some halucination. The last response and the ones I was getting constantly an hour earlier were complely unrelated to the contents of the file.
For some reason the same assistant with exactly the same configuration used in the Playground either works much better or just fails the run.
I thought that maybe implementing a 30 second wait between the upload of the file and the message call will make a difference but it didn’t.