Hi everyone,
I am developing an AI-powered application for contract management, designed to answer user questions by fetching relevant contract information.
How It Works:
- A background process reads contract information and stores it in an SQLite database.
- Some functions retrieve structured information from the database, while others fetch the entire contract text and pass it to an LLM along with the user’s question.
- I plan to implement the following functions to assist the LLM in making the right function calls:
Planned Functions:
- GetAnswerUsingSQL – Uses a database query to retrieve contract information from one or more contracts.
- CompareContracts – Retrieves the full text of two or more contracts to compare differences.
- SummarizeContracts – Summarizes or provides an overview of one or more contracts.
- CheckContractCompliance – Checks contracts for compliance issues and red flags.
- QueryContract – Retrieves and queries a single customer contract.
Key Rule:
I want the function GetAnswerUsingSQL to be used only when the question involves fetching structured data from multiple contracts, as some contract details are already stored in the SQLite database.
Advice Needed on Key Implementation Questions:
1. How to Inform the LLM About Available Functions?
I currently provide:
- A system prompt describing the available functions and when they should be used.
- Function descriptions along with their parameters.
- Examples of correct function usage.
Is this approach correct, or should I rely only on function descriptions and parameters without additional examples in the system prompt?
2. Handling Multiple Function Arguments?
Example User Query:
“Can you point out the important differences in the contracts of Customer ABC and DEF?”
- The LLM correctly identifies the need to call CompareContracts.
- However, it sometimes only returns one customer (e.g., “ABC”) instead of both ABC and DEF as function arguments.
How can I ensure the LLM consistently extracts all relevant function arguments from user queries?
3. Managing Conversation History When Context is Huge?
- The answer and contract text can be very large, so I plan to only store the user’s question and details of the function call suggested by the LLM.
- If I pass this as conversation history, will it help the LLM maintain context in future interactions?
- Is there a better approach to handle conversation history efficiently while keeping memory usage manageable?
Looking for Expert Guidance
I would appreciate any expert advice on these points and any additional best practices related to function calling in an LLM-powered application.
Thanks in advance!