GPT-3 Fine-Tuning for Keywords Extraction

Community
1

Hello everyone !

You’ve probably heard about our projects with Mojo at Hoomano. Here is the last article on our blog, dealing with our first try with fine-tuning. We fine-tuned a curie model to extract keywords from news article. With only 100 labelled data the results are really good for us and we’re glad to share this success with you :partying_face:

Some examples of keywords generated by our fine-tuned model:

“Is sugar the main cause of diabetes? Not really — Malaysian expert points to factors beyond sugar. KUALA LUMPUR, Dec 13 — Diabetes is a disease characterised by high blood sugar levels. Hence, many may wonder whether it is caused just by eating sweets. Although it may be true that having large amounts of sugary food daily may increase one’s risk of diabete…” => “diabetes”

“Maroc-Algérie: quand le foot transcende les tensions politiques. ““Ce soir c’est le sport qui parle””, s’exclame Zakaria, un supporter marocain à Casablanca. Pourtant, le quart de finale de la Coupe arabe de football entre le Maroc et l’Algérie – remporté par cette dernière aux tirs au but (2-2 à la fin du temps réglementai…” => “football”

5 Likes
2

Super cool result. Great work on this!

3 Likes
3

For refining down to a single keyword, I would bet that Babbage could do just as good, but be even cheaper. You might consider running a trial again one of the smaller engines.

1 Like
4

Adding on to what @daveshapautomator said, you can now bootstrap the development by using our own curie finetuned model to generate a set of 1000 labelled data points (as an example), and use a smaller engine.

1 Like
5

I was not convinced until now by Babbage performance, but I love this idea !! Thank you very much, I’ll give it a try and let you know !

1 Like
6

I meant *your* own curie model. :slightly_smiling_face: @kelly.roussel