Seeking Advice on Function Calling

Hi everyone,

I am developing an AI-powered application for contract management, designed to answer user questions by fetching relevant contract information.

How It Works:

  • A background process reads contract information and stores it in an SQLite database.
  • Some functions retrieve structured information from the database, while others fetch the entire contract text and pass it to an LLM along with the user’s question.
  • I plan to implement the following functions to assist the LLM in making the right function calls:

Planned Functions:

  1. GetAnswerUsingSQL – Uses a database query to retrieve contract information from one or more contracts.
  2. CompareContracts – Retrieves the full text of two or more contracts to compare differences.
  3. SummarizeContracts – Summarizes or provides an overview of one or more contracts.
  4. CheckContractCompliance – Checks contracts for compliance issues and red flags.
  5. QueryContract – Retrieves and queries a single customer contract.

Key Rule:
I want the function GetAnswerUsingSQL to be used only when the question involves fetching structured data from multiple contracts, as some contract details are already stored in the SQLite database.


Advice Needed on Key Implementation Questions:

1. How to Inform the LLM About Available Functions?

I currently provide:

  • A system prompt describing the available functions and when they should be used.
  • Function descriptions along with their parameters.
  • Examples of correct function usage.

:point_right: Is this approach correct, or should I rely only on function descriptions and parameters without additional examples in the system prompt?

2. Handling Multiple Function Arguments?

Example User Query:
“Can you point out the important differences in the contracts of Customer ABC and DEF?”

  • The LLM correctly identifies the need to call CompareContracts.
  • However, it sometimes only returns one customer (e.g., “ABC”) instead of both ABC and DEF as function arguments.

:point_right: How can I ensure the LLM consistently extracts all relevant function arguments from user queries?

3. Managing Conversation History When Context is Huge?

  • The answer and contract text can be very large, so I plan to only store the user’s question and details of the function call suggested by the LLM.
  • If I pass this as conversation history, will it help the LLM maintain context in future interactions?
  • Is there a better approach to handle conversation history efficiently while keeping memory usage manageable?

Looking for Expert Guidance

I would appreciate any expert advice on these points and any additional best practices related to function calling in an LLM-powered application.

Thanks in advance!

1 Like

Hi @ramanan.iyer !

Just as a general tip, this documentation from OpenAI is an excellent guide on best practices when it comes to function calling.

Looks good to me!

Have you tried using a list, i.e.

def CompareContracts(customers: list) -> str:
    ...

You may want to create an additional call to distill/summarize the returned text, or extract some kind of entities and relations - essentially anything that provides a more compressed context of the retrieved text, and then use that in subsequent calls. You can try some kind of recursive summarization so that the context history doesn’t just grow indefinitely. This is most likely what OpenAI actually uses in ChatGPT itself.

OpenAI also recommends trying out finetuning for specific function calling, if all else fails - there is a solid cookbook on that here.

1 Like

Hi @platypus
Thanks a lot for your feedback.

  • Regarding point 2, yes I have defined a list.
  • Regarding managing conversation history, thanks for the suggestion I will try it out. I may have issues even after trying to summarize the answer because at times the answer may involve information from multiple contracts.
  • Is it OK if we don’t keep the context in the conversation history?

Kind Regards
Ramanan

1 Like

Re: conversation history, generally it’s ok, depends on how the user interacts with the system. In all the systems I’ve seen, the user expects multi turn conversations and for the previous messages to be taken into account

Hi @platypus
You are right regarding conversation history. In my case it could be a little different that the User may ask questions regarding Contract details about one Customer and then about another Customer. So in this case the Conversation history may not be so important.
However, it could also turn out that they may ask additional questions about the Contract for the same Customer. So have to find a way of balancing it :slight_smile:

Kind Regards
Ramanan

1 Like