Is fine tuning quite limited?

API
1

I was excited about using fine-tuning to essentially achieve unlimited size in context.

However according to the doc (OpenAI API), only these base models are available for fine tuning:

davinci , curie , babbage , and ada

I tried these base models in play ground and they seem very incoherent. Can you really build production-level app by fine tuning these models?

Yi

1 Like
2

Welcome to the community. Fine-tuning does have its limitations. It’s still useful in some cases, but a lot of people are switching to embedding + completion… If you search around the forum, you should find some useful for posts.

2 Likes
3

Thanks, I’ll search around and see what I can find

4

Also, is my understanding correct that fine tuning can only be used on base models such as davinci, curies, and since they lack instruction, it’s hard to get good results with them? Versus using the embedding + completion approach, I get to use models like text-davinci-003, which has been instructed and allows me to build something closer in quality to GPT 3.5 more quickly.

5

Yes, correct.

Depends on the use-case, the dataset, and the fine-tuning performed, but generally, yes, it’s better to stick with the newer models and embedding.

You can also get access to GPT-3.5-turbo (which is really cheap now at $0.002/1000 tokens…), and GPT-4 is in beta and there’s a waitlist.

But yeah, most would agree, I think, that embedding + completion endpoint is the way to go moving forward.

Hope you stick around our growing community here! We’ve got a great bunch of really smart people - and me! Small smile.

2 Likes
6

Paul,

Great, thanks for your input. Yes, it’s quite exciting to be a part of this community. I look forward to participating with you all.

1 Like
7

Hi Paul, how embedding can help for more specific financial categorisation based on business entity historical data?