Train model based on 6 years of tech support tickets

GPT builders Plugins / Actions builders
,
1

Hello,
We get repeat support tickets all day long. Currently we answer most of them with canned responses. I would love to be able to train GPT on the 6 years of support tickets we have and have it answer the new customer tech support tickets that come in. I am having problems figuring out how to train the model.

Is the best way to do this by fine-tuning or creating a custom GPT?

As well, any thoughts on how to prepare the data? 6 hears is a lot of support tickets…

2 Likes
2

another way is to use Retrieval-Augmented Generation (RAG).
to prepare your 6 years of data for RAG, check the Use cases section in Embeddings page for an example. also check Text search using embeddings in the same page for the sample way how to search through the documents.

3

You have a nice problem, because you already have everything you need to create the solution.

It is not necessary to fine tune a model for your use case, you can build an effective solution utilizing a basic vector database approach.

Here is how it will work.

I assume your data is in the approx format:

Customer support query
Support response

To get from customer query to response currently, I assume you need to check docs or the info may reside in the heads of your support staff.

So it is necessary to clone this process using a hybrid approach, as follows.

  1. All of your previous customer queries and responses can be batch processed through a prompt to provide a summary of the query:response pair, for example:

“Respond to a customer enquiry about ABC by providing details about A, B, and C”

When you have summaries you can store them in a vector database along with the link to the id of the original query and support response details.

  1. All of your support documentation will be stored in a vector database along with the link to the specific page/section.

How it will work in action:

A new support request comes in:

  1. System searches through both support docs vector db AND previous support response summaries vector db.
  2. System finds closest matches in both vector db’s, passes the 10 closest matches to a re-ranking prompt (to select most relevant) and then uses the filtered info to help guide the response to the query, which will include links to the relevant info in your support docs.

If customer asks for additional info then the system goes back through the loop.

As each support request is answered by the system the support request/response summary can be added to the vector db to expand the info available for future system requests.

This is obviously a high level overview but is an exact description of a similar system I have already implemented.

If you need clarification of any steps then lmk.

1 Like
4

RAG is the way to go. Custom GPT is also RAG behind the scenes.

Finetuning is great for style but not as good for new data because OpenAI finetuning has low rank.

To finetune a model with your own data you may want to use local model, such a Llama3, as the foundation. Of course, the cost will be much higher and require hardware and more technical work.