High accuracy chatbot trained on our docs - not happy with the responses

GPT builders
1

Hi everyone,

this is my first post on this forum. I need a bit of advice on the approach of how to create accurate chatbot trained on our data. My current approach is:

  1. export pages / files from our docs
  2. store them as files in open ai and attach to a vector store
  3. create assistant with a file_search function. Model 4.1 nano / mini
  4. test, test, test
  5. result: lot of halucinations e.g. I am asking what is our latest release and it’s features. Releases are stored as separate files / pages with detail information for each LTS release.

I am not an AI / ML expert so bit of guidance on how to approach this problem would be helpful. Thank you,
j.

2

Hi and welcome to the dev forum.

Personally, I wouldn’t go with vector store/assistant API, especially if precision is a must.

I would start by defining my relational database with all the data they bought needs to have access to.

I would add procedures to embed the stored data to allow access by vector search.

Then I would wrap that database in a rest API and define access to the information via tools.

Then I would use responses API for the bot.

3

Have you tried with gpt-4.1 regular version?

I would recommend playing with temperature and top_p, specifically lowering those in small increments, to see if this improves your results.

Also, prompt engineering is an overlooked aspect of LLM integration and may actually be a major factor of these issues. OpenAI has a good resource on prompting 4.1: GPT-4.1 Prompting Guide

How is this any different than OpenAI’s vector store?

1 Like
4

Personally for me having full control over the whole system and each of the elements in it allows to get way better precision and exact fit into my apps. More work in the beginning but far better results and less work in the long run, especially if planning to scale/reuse.