Comparison of GPTs to Assistant API

API Feedback
1

Hey there,

i am currently creating a chatbot for our website with the Assistant API. In parallel i am working with a self designed GPTs for my development work. I gave both, the Assistant and the GPTs the same PDF sources.

The answers of my GPTs are awesome and it takes specific knowledge of my PDFs and generates perfectly matching answers.

When i try exact the same questions on my Assistant, he gives more general answers and doesn’t include the knowledge of the uploaded PDF’s.

Specific example:
I uploaded a PDF with specific offset values i need to configure, when integrating my sensor system into a robot. The GPTs included these exact offset values in the answer, but the assistant is not able to use these values for answering.

Assistant Configuration:
I tried gpt-4, gpt-4 0613 and gpt-4 1106-preview.
Only with the model gpt-4 1106-preview i am able to enable Retrieval.
Am i right that i need retrieval, that the Assistant uses my in the configuration panel uploaded PDF files or does this only refer to the files uploaded in the chat?

All of them are not giving the same quality of answers as the GPTs. gpt-4 turbo is not provided in my settings panel.

Does someone have suggestions if i am doing something wrong? In some forum articles people are saying it’s the other way around, the assistant gives much better answers.

1 Like
2

Are you using the same system message that was generated for the GPT? You could also try out the code interpreter in comparison to see if you can get the relevant information better this way.
It is also possible that your GPT extracts the data and converts it to a different format.

GPTs are a bit of engineering in the background that you have to do yourself here :slight_smile:

3

I have a feeling that in case of chatGPT service there’s a hidden layer of prompt and parameters optimization. If it’s true, that might be a reason of different output, even if we are not using agents (GPTs and Assistants). And with agents more is happening under the hood, so even a small change in, for example, temperature, might result in big differences in output.
BTW. For me retrieval in Assistants just doesn’t work…

1 Like
4

Thank you for your feedback.

Somehow it works now. I uploaded the PDF and have set the used Model to gpt-4 1106-preview.

Now he is extracting the values of my pdf and outputs them in the answer correctly. I think the quality of the answer of GPTs is still a bit higher, but at least im getting the information now.