Assistant with PDF in Vector Store delivers worse answers than ChatGPT with same PDF

API
1

I have uploaded a 70mb german PDF to a vector store. It is a rule PDF of a role playing game.
Then I associate an assistant and ask questions in the playground. I use a system prompt that assistant should only use the pdf and not make anything up. I asked 4o-mini three questions and all answers were wrong/hallucinated. 4-mini hasnt produced better results.

In comparison I have tried ChatGPT and use the same file by onedrive connection. I have got much better answers there.

How can I fix this in my assistant?

2

Welcome the best AI developer community online!

Is the PDF images or converted text? If it’s having to do OCR, it might be as good.

Also, what model are you using? What’s your prompt(s) look like?

More details can help the community help you!

Again, welcome aboard!

3

Thank you for your nice welcome.

I have created another assistant with the same PDF in english language.

In https://platform.openai.com/playground/assistants

The text in the PDF can be copy/pasted in acrobat reader. So no image text. The PDF contains images, but they are just artwork.

First I have tried gpt-4o-mini
Then gpt-4o

system content: “You are a helpful assistant for a vampire role playing game. Dont make something up which is not in the provided text. If you dont know tell the user”
user content: “what is obfuscate 3?”
or
user content: "how much blood does obfuscate 3 cost3?

I have the PDF connected by a vector store
I have tried 3, 10, 30 search results

Other settings:

temperature: 0.55
top p: 0.55

I have reduced the values since I want rules not a fantasy story.