'Intent' Feature for memory and query assistance in Azure ai search and open ai integration

API
,
1

I am interested in understanding the logic behind the intent feature in the integration between OpenAI’s chat completions and Azure AI Search. Specifically, I noticed that when appending and sending chat history for retrieval-augmented generation (RAG), there is a feature in the response named “intent.” This output appears to be a reformulated query that is sent to the AI search to ensure it receives a meaningful request, allowing the search engine to complete its task effectively.

Take a look:

completion = client.chat.completions.create(
    messages=[
        {"role": "user", "content": "Who is the first man to land on the moon?"},
        {"role": "assistant", "content": "The first man to land on the moon was Neil Armstrong."},
        {"role": "user", "content": "How old was he at that time?"}
    ],
    model=deployment,
    extra_body={
        "dataSources": [
            {
                "type": "AzureCognitiveSearch",
                "parameters": {
                    "endpoint": os.environ["SEARCH_ENDPOINT"],
                    "key": os.environ["SEARCH_KEY"],
                    "indexName": os.environ["SEARCH_INDEX_NAME"],
                }
            }
        ]
    }
)

Not Only the chat completion returns the correct answer which is 38 years old but also return an ‘intent’ feature :

print(completion.choices[0].message.context['intent'])

["How old was Neil Armstrong when he landed on the moon?", "What was Neil Armstrong's age when he landed on the moon?", "How old was Neil Armstrong when he was on the moon?"]

I’m very interested in understanding this mechanism, as I’m working with a stateless LangChain agent, and I would love to implement something similar.

I would like to know what prompt OpenAI uses to reformulate the query and send it to Azure AI Search using the “intent” feature. I want to be able to replicate this functionality using a prompt or some tools.

I am looking for a solution that allows me to summarize user inputs and send those reformulated queries to a LangChain agent. Is there any documentation or report that explains how to implement this intent feature in a standard chat completion? Additionally, could you point me to any prompts used by OpenAI to reformulate and query the questions?

I would greatly appreciate any guidance or resources you could share. Thank you so much for your help!