Assistants API Summarizing Documents In Native Language

API
, ,
1

I am using the Assistants API to summarize documents. Many of my users upload documents in non-English languages such as Korean, Spanish, German, etc.

I am struggling to write instructions for the Assistant and a message for the thread that consistently summarizes the documents in the same language in which it was written. Often, the summaries end up defaulting to English when the documents are not written in English. Today, a completely English language document was summarized in German, presumably because the word “German” appears in the text of the document.

This is what the code looks like when I create the Assistant:

assistant = client.beta.assistants.create(
            name=f"{name}",
            instructions=f"""
                
            You are an AI summary Assistant. Go step by step because the result of the first step serves as input to the next steps.

            1. Detect the primary spoken language used in the documents.  
            2. Take extreme care to write a summary of the document in the detected spoken language, also answering any questions asked by the user.

            Please maintain the context and key points from each document while keeping the summaries concise yet detailed. Take your time and provide a direct summary by pulling out the most important key points, facts and opinions.  Use 2000 words or less.""",
            tools=[
                {
                    "type": "retrieval"
                }, 
            ],
            model=model
        )

And here’s what my thread creation looks like:

thread = client.beta.threads.create()
content = "You have the files in the Assistant. Summarize this document in the language in which it was written. Use 2000 words or less."
run = submit_message(assistant.id, thread, content)

I have tried innumerable variations of the instructions and prompts. I’ve tried only providing instructions to the assistant and omitting instructions in the thread. I have tried giving the Assistant pseudo-code to follow. I have tried splitting the messages in the thread into two parts: one that asks for language identification and then another that says “use that language” for the summary.

Nothing I have tried has worked consistently.

Any suggestions and/or commiserations would be greatly appreciated. :slight_smile:

2

The AI is inherently a chat model. It will respond primarily in the language of the user and respect the user input more. So if you can change the user’s instruction to the destination language, that may work far better. A bit of document language detection and pretranslated instructions?

3

Also, which model and params are you using?

4

Sorry, I don’t see where one specifies params for use with Assistants. I am using gpt-4-1106-preview, which I think is the only model that supports Retrieval.

1 Like
5

My bad, there are no model params available for now in Assistants API. gpt-4-1106-preview is normally following instructions pretty good. To debug, need to look into the whole messages thread.

6

I would translate all of my prompts to the native language of the upload, and then explicitly ask that output be in that same language.
You could possibly do this on-demand and automatically.

For example, you could start with the initial document, and ask “what language is this in? output only the name of the language, no explanatory text.”
Then, check whether you have a translation remembered (in a database) for that language for your prompts.
If not, send each prompt to the system, saying “please translate this prompt to {language}” and remember the translated prompt for next time.
Finally, translate the phrase “please output the summary in {language}” as well and put that at the end of your prompts.
Then, provide that prompt in the appropriate language, together with the documents, and it should (fingers crossed) stay in that language.

1 Like
7

You were indeed correct.

Previously, I was allowing users to upload documents with no additional text input. Now I am requiring that they give directions in their desired output language.

So, for example, users can now upload English-language documents, but give additional text input in Chinese and my service will generate summarized PowerPoint slides in Chinese based upon the documents. :slight_smile:

1 Like