Hello,
I am trying to fine tune davinci based on a blog, and my goal is to have it be like a writing assistant when writing the blog, using a similar style. My approach was to scrape each paragraph, have the prompt be the first sentence of the paragraph, and the completion be the entire paragraph text. Here is a sample of what I ended up with.
{"prompt":"You may have missed a big financial story earlier this month (Not FTX) ->","completion":" You may have missed a big financial story earlier this month (Not FTX). On thesame day that the Dow fell over 600 points, and Bitcoin plunged to its lowestlevel since 2020 Microsoft Co-founder Paul Allen's art collection fetched arecord $1.6 billion worth of art in less than 24 hours. The super-wealthy seeart as a safe investment amid a tumultuous and uncertain global economy, perBBC.\n"}
{"prompt":"Three things to know about the asset class: 1) fine art can help protectpurchasing power when inflation is high (Goldman), (2) fine art has a -0 ->","completion":" Three things to know about the asset class: 1) fine art can help protectpurchasing power when inflation is high (Goldman), (2) fine art has a -0.34correlation to equities (Citi), and 3) contemporary art has appreciated 13.8%annually on average for the last 26 years.\n"}
{"prompt":"So why am I telling you this if only the ->","completion":" So why am I telling you this if only the .01% wealthiest people can capitalizeon this asset class? Well, now you can too, with Masterworks.com[ fractional art investment platform. \n"}
{"prompt":"All offerings are SEC qualified (see offering circulars on the site), and theirtrack record is solid so far…their last three offerings handed members 17 ->","completion":" All offerings are SEC qualified (see offering circulars on the site), and theirtrack record is solid so far…their last three offerings handed members 17.8%,21.5%, and 33.1% annual net returns, respectively.1[\n"}
{"prompt":"Happy Monday! This is an essay I’ve been thinking about and working on for alittle while – I quote myself a lot because I’ve been unintentionally writingabout the theme since the beginning of Not Boring – and I’m really excited toshare it with you ->","completion":" Happy Monday! This is an essay I’ve been thinking about and working on for alittle while – I quote myself a lot because I’ve been unintentionally writingabout the theme since the beginning of Not Boring – and I’m really excited toshare it with you. It’s as close to a Theory of Everything as I can come up withto understand the world in 2022. \n"}
{"prompt":"Once I started thinking about the world this way, I’ve seen it pop upeverywhere, from politics to energy to AI ->","completion":" Once I started thinking about the world this way, I’ve seen it pop upeverywhere, from politics to energy to AI. Maybe it’s Baader-Meinhof[ it’s real. \n"}
{"prompt":"And as I’ve tried to explain it to people, I’ve realized that I just need towrite down as coherent a thesis with as much evidence as I can, and open it upto get feedback and pushback ->","completion":" And as I’ve tried to explain it to people, I’ve realized that I just need towrite down as coherent a thesis with as much evidence as I can, and open it upto get feedback and pushback. If nothing else, I hope it can be an interestingconversation starter at Thanksgiving dinner on Thursday.\n"}
{"prompt":"The idea is this: the world oscillates between centralization anddecentralization, with progress sloping upward through the turns ->","completion":" The idea is this: the world oscillates between centralization anddecentralization, with progress sloping upward through the turns. We’reapproaching an era of decentralization.\n"}
{"prompt":"You should not read this as a bull case for crypto ->","completion":" You should not read this as a bull case for crypto. While the word“decentralization” is tied to crypto, this piece is broader than that. Iactually think one of the things I’ve gotten wrong about crypto is that it’sa symptom of a larger trend towards decentralization, not the cause.\n"}
{"prompt":"When our descendants look back on our era from a thousand years in the future,they’ll view the way that we do everything as incredibly, laughably primitive ->","completion":" When our descendants look back on our era from a thousand years in the future,they’ll view the way that we do everything as incredibly, laughably primitive.Our most advanced technology, best-laid plans, and most resilient forms oforganization will be viewed, if they’re discussed at all, as historicalartifacts of a time before we knew what the hell we were doing. \n"}
Now it’s not perfect, as I don’t have the exact blog text, and I’m relying on scraping it. I had around 2K prompts like this.
After finetuning, the results were rather dissapointing. The AI had trouble making clear sentences, and sometimes just returned gibberish letters.
Do you think there is any better approach for training an AI from a lot of text? Or do I just need more data, or more correct, complete sentenced data so the AI can learn better?
Thank you.
