So I was bored one day and I stumbled upon the fine tuning feature. And I thought “hey, what if I try to make my bot more social by feeding it conversations between friends?”, to say the least, it was a cool 3 am idea and I wanted to share the lessons learned.
1. Models based on sole messages
So I had several and I still going forward with this, but if you want to simulate specific people you are better off with the format of {"prompt": "username","completion": "their message"}. This yields the best results for random individual messages. It is not for conversations.
Using this I created two small commands, a game and a random saying generator:
Also settings I found to work better, were high batch sizes, multiple epochs with a not steep learning multiplier. And when it comes to querying the higher the temperature the more sensible the result is? As well as a high freq penalty is needed because of emojis.
The game, where you have to guess who would have said the predicted algorithm:
The saying generator, where the numbers is the temperature for each message:
2. Models based on multiple messages:
So for this one, I basically build the dataset with prompts going by the style of
{prompt: "username1: msg1",completion: "username2: reply1"}
This has fairly decent results, as you can see below:
I’m still experimenting around with fine tuning for better results, but this is not where the bot stops at all.
The bots primary focus is education, the social part is just for researching around. For example it can answer questions such as:
It supports various subjects, although it struggles a bit with maths. Regardless, it also can pinpoint exact questions from A-Level past papers. For example:
Aside from these features, the bot is also capable of providing help for various subjects through multiple ways, such as suggesting how to improve essays, generate questions about specific topics and more!
On its social aspect, aside from being able to converse normally about any topic, it can also start new ones if it sees activity is low. For example a conversation starter is:
Let me know what you think of it.