Can I combine Embeddings with Finetuning to develop a bot?

API
,
1

Hello community,

I am working on a project to develop a bot that can generate code snippets for structural engineering software such as Sofistik Teddy. I have some understanding of embeddings and fine-tuning, but I am not an expert. I am also learning API development.

I tried using ChatGPT-4, but it did not produce accurate code. I then tried to train the model in chat window with some examples and detailed code explanations, but it was very time-consuming and there are many possible variations of examples. Sofistik itself has some good examples with self-explanatory comments, which I think GPT-4 can understand.

However, I wonder if there is a better way to train the model. Instead of manually converting every example into a GPT-4 prompt, can I use the Sofistik help document(pdf, explains syntax and keywords of teddy code)and examples as embeddings and teach the model? That way, the model can reason through the document and examples to answer my questions with accurate code.

Is this possible? Any advice or suggestions would be appreciated.

image
image1920×983 181 KB

2 Likes
2

I need to do something like this too. Are there any updates?

3

You cannot fine-tune gpt-4 currently. It is likely better at any code writing than a fine-tune could be, unless you invent your own programming language and want to train on 10000 task examples.

supplementary embeddings methods for database augmentation can still work the same with your model that has refined behaviors.