Should I use a vector store or should i fine-tune?

Hey im new to developing with openAI’s API and I wanna create a small project of a high school’s english teacher’s assistant app to better understand the API.

I want each english teacher in my school to be able to assign 1 or more tasks of having a conversation about a certain poem or chapter from a book in the study material of it’s class (and students should be able to complete the task by having a conversation led by an assistant designed for the class).

  1. Should it really be an assistant per class? or maybe per task? How do i design it correctly?
  2. Suppose there is one 200 pages book that accompanies the students throughout the semester. Should I add the whole book to the assistant’s vector_store and with each task raised, prompt it to focus the conversation on so and so chapters? because I assume that would highly damage the responsiveness of the bot, but on the other hand I don’t imagine it’s possible to fine tune a model to only 1 book and make it an expert solely on that content
1 Like

Welcome to the community!

Let’s start with your question:

Should I use a vector store or should i fine-tune?

This is not really an either/or question.

Fine-tuning is good for Setting the style, tone, format, or other qualitative aspects (https://platform.openai.com/docs/guides/fine-tuning/common-use-cases)

You can think of fine tuning as acting lessons, when you want to give fewer director’s notes in your prompt.

On the other hand, Vector Store objects give the File Search tool the ability to search your files. (https://platform.openai.com/docs/assistants/tools/file-search/vector-stores)

Vector stores on their own is just a personal google that works just for your files.

Another related concept is Embeddings: OpenAI’s text embeddings measure the relatedness of text strings. (https://platform.openai.com/docs/guides/embeddings)

Embeddings are (probably) the underlying technology behind vector stores, but you can use them yourself raw - many people are using that for Retrieval Augmented Generation - looking for salient documents that are relevant to the topic and injecting them directly into the prompt.


Given all that, I’d just like to mention that if your book is a common book that was published a while back, there’s a good chance that it was part of the training data - that the model knows about it. It might take some finessing with the copyright guard**** mechanisms but it’s possible that you might be able to get away with not using vector storage at all.

Good luck!

4 Likes

Thanks so much!
So if maybe this is a school specific work book that isn’t accessible on-line and i would want to allow the teacher to assign a practice that is about certain pages - would I probably prefer to add the whole book to the vector store and prompt the model to stick to those pages at the beggining of the session? If so, will it affect run-time? because i’d really care about that haha

If you can afford it, you could consider just sticking the relevant pages (and necessary ancillary information) into the context (e.g. system prompt)

That sounds nice to improve performance. Could you maybe refer me to how exactly is that done?

If you go here you can try it out:

https://platform.openai.com/playground/chat?models=gpt-4o

in the top left, there’s this </> icon you can click, where it shows you the code :slight_smile:

1 Like