Great to see you’re getting the results you’re looking for! Please share any updates when you have some.
Interesting approach and thank you for sharing with such detail! I’ver never personally had good results with markdown but using clear delimiters always works wonders. Will need to try the way you’re implementing it.
Have you had any issues while the amount of documents grow (inaccurate answers, slower answers, etc)? Keeping each to 400 tokens or less.
Hi @diego.cardenas, I want to try this approach as well, I have a external endpoint that retrieve an information of a user base of an id as a parameter, the funny thing is that the gpt hallucinates big time, the gpt response well from the external api but the data that the gpt display is garbage data not the data that suppose to fetch from the API.
I tested in postman and the external api deliver the data correct.
How can I use your approach to make this work?
Take a look of the schema
{
“openapi”: “3.0.1”,
“info”: {
“title”: “Resource Request using an ID”,
“version”: “1.0.0”
},
“servers”: [
{
“url”: “https://mydomainblahblahblah.com”
}
],
“paths”: {
“/api/json/reply/”: {
“get”: {
“summary”: “Get Information from using an id parameter”,
“operationId”: “resourcerequest”,
“parameters”: [
{
“name”: “id”,
“in”: “query”,
“required”: true,
“schema”: {
“type”: “string”
}
}
],
“responses”: {
“200”: {
“description”: “Successful response with the resourcerequest”,
“content”: {
“application/json”: {
“schema”: {
“type”: “object”,
“properties”: {
“Name”: {
“type”: “string”
},
“Id”: {
“type”: “integer”
},
“UserLogin”: {
“type”: “string”
}
}
}
}
}
},
“400”: {
“description”: “The user does not exist, or you do not have access to it.”
}
}
}
}
}
}
Hey! @diego.cardenas, thanks for the answer. Unfortunately, I didn’t succeed with providing full data to assist via files API regardless of formatting. I switched to providing assistant with functions that will do the search on DB side. Also, converting DB to vector format really helps. Also, there are vocabulary trees…
Quick update:
I’ve tried @stevenic 's approach and it’s been working fine for now. Using the Assistant API with retrieval. Same as before, using TXT file has been the most efficient and providing constant results, this hasn’t changed. What I’ve done is the following:
- Each document, follows the format for every item:
### Name: Steve
Age: 55
Title: Founder
Basically, there is a delimiter “###” with a relevant title and the rest
of the content below with relevant parameters.
2. Each document, regardless of it’s size (maximum by OpenAI is 512 MB), has each item which follows the format from point 1 is maximum 400 tokens.
3. Following the updated documentation:
How it works
(https://platform.openai.com/docs/assistants/tools/how-it-works)
The model then decides when to retrieve content based on the user Messages. The Assistants API automatically chooses between two retrieval techniques:
- it either passes the file content in the prompt for short documents, or
- performs a vector search for longer documents
Retrieval currently optimizes for quality by adding all relevant content to the context of model calls. We plan to introduce other retrieval strategies to enable developers to choose a different tradeoff between retrieval quality and model usage cost.
This was assumed before but depending on the size, the complete content of the file is passed to the model or a vector search is performed and the result is passed to the model. What I’m understanding from @stevenic recommendations and the OpenAI documentation is that each item that we have in all our documents is embedded. So we should add enough context in the item for the vector search to be efficient and taking into account costs of course.
Hello
I m using assistant api with retrieval and function calling. And have also attach a json file with purpose as assistants.
- My requirement is to get most relevant/similar objects from the file for the prompt given. (min 10 and max 100)
- the file has 1000 objects, simple objects not nested more than 1 level.
the assistant is giving good result but its not giving the mentioned quantity.
It gives some where between 1-6 objects, even after mentioning in the instructions for min 10 and max 100.
the assistant doesnt goes though all the 1000objects, if it get 2-4 matching objects from the top 10-20 objects, it return them
any help here?
am i taking the correct approach ?
Well getting no incorrect values and only a lack of quantity using GPT-3.5 is an achievement! Can you share you prompt and the file format you’re using? I mean content + extension of the file.
Hy,
i m using gpt-4-turbo-preview.
the file is a json file, it has list of object.
object: {
id:string;
field1: {subField1:string, subField2:string};
field2:string;
field3: string[];
field4: number;
field5:string;
field6: string[];
}
I have add an instruction, while creating the assistant.
where i have explicitly mentioned to return min of 15 object.
instruction:
You are an expert in identifying suitable objects, from the knowledge base, for the given query
Use your knowledge base to best respond to the query, go through all objects before responding.
You have to provide a maximum of 100 objects and a minimum of 10 objects for any query, if you feel no object matches the requirement, try to extract objects that has matching field2. For each object you select, you have to give a reason for the selection and a match percentage describing how similar the object is for the given query.
i tried to convert the object into
### id 1
field1: string
field2: string
field3:string
field4: string
### id 2
field1: string
field2: string
field3:string
field4: string
and created a .txt file and uploaded it to openai files.
while creating a assistant with retrieval and mentionin the fileId i get an error
Error: 400 Failed to index file: Unsupported file file-id type: application/octet-stream error_code: unhandled_mimetype
My bad, i am getting incorrect results.
Assistant is creating custom objects.
Hey @roopak , did you solve this?
Nope, no luck.
any help will be very much appreciated
Hi @diego.cardenas your solution partial works for me, but i think today, Jan 2025, there are another news technics to create better files to vector search.
Can you share your news hacks after that?
Very Best!
I’ve seen better results when structuring product data as a CSV or SQLite database instead of raw JSON. That way the assistant (with Code Interpreter enabled) can query/filter more reliably for categories and attributes.