Please review my hyperparameters and prompt for Fine-tuning

Hi, I’m trying to fine-tune a model to give Kurzgesagt (scientific channel) YouTube video ideas, but I’m not very familiar with ML, so if somebody could review my JSONL and hyperparameters.

My JSONL:

{“prompt”:"",“completion”:“What Dinosaurs ACTUALLY Looked Like?”}
{“prompt”:"",“completion”:“Two Chapters From Our New Book – Exclusive Preview!”}
{“prompt”:"",“completion”:“Can YOU Fix Climate Change?”}
{“prompt”:"",“completion”:“This Virus Shouldn’t Exist (But it Does)”}
{“prompt”:"",“completion”:“How The Immune System ACTUALLY Works”}
{“prompt”:"",“completion”:“The Largest Black Hole in the Universe - Size Comparison”}
{“prompt”:"",“completion”:“How To Terraform Venus (Quickly)”}
{“prompt”:"",“completion”:“The Day the Dinosaurs Died – Minute by Minute”}
{“prompt”:"",“completion”:“What Are You Doing With Your Life? The Tail End”}
{“prompt”:"",“completion”:“TRUE Limits Of Humanity – The Final Border We Will Never Cross”}
{“prompt”:"",“completion”:“What If You Fall into a Black Hole?”}
{“prompt”:"",“completion”:“Do we Need Nuclear Energy to Stop Climate Change?”}
{“prompt”:"",“completion”:“What if the World turned to Gold? - The Gold Apocalypse”}
{“prompt”:"",“completion”:“Worst Nuclear Accidents in History”}
{“prompt”:"",“completion”:“What if We Nuke the Moon?”}
{“prompt”:"",“completion”:“Can You Upload Your Mind & Live Forever?”}
{“prompt”:"",“completion”:“What If Earth got Kicked Out of the Solar System? Rogue Earth”}
{“prompt”:"",“completion”:“How Large Can a Bacteria get? Life & Size 3”}
{“prompt”:"",“completion”:“Geoengineering: A Horrible Idea We Might Have to Do”}
{“prompt”:"",“completion”:“When Time Became History - The Human Era”}
{“prompt”:"",“completion”:“Is It Too Late To Stop Climate Change? Well, it’s Complicated.”}
{“prompt”:"",“completion”:“The Largest Star in the Universe – Size Comparison”}
{“prompt”:"",“completion”:“The Warrior Kingdoms of the Weaver Ant”}
{“prompt”:"",“completion”:“Unlimited Resources From Space – Asteroid Mining”}
{“prompt”:"",“completion”:“What Do Alien Civilizations Look Like? The Kardashev Scale”}
{“prompt”:"",“completion”:“What Is Intelligence? Where Does it Begin?”}
{“prompt”:"",“completion”:“Who Is Responsible For Climate Change? – Who Needs To Fix It?”}
{“prompt”:"",“completion”:“Could Solar Storms Destroy Civilization? Solar Flares & Coronal Mass Ejections”}
{“prompt”:"",“completion”:“The Past We Can Never Return To – The Anthropocene Reviewed”}
{“prompt”:"",“completion”:“Why Are You Alive – Life, Energy & ATP”}
{“prompt”:"",“completion”:“The Coronavirus Explained & What You Should Do”}
{“prompt”:"",“completion”:“Why Blue Whales Don’t Get Cancer - Peto’s Paradox”}
{“prompt”:"",“completion”:“How to Make a Kurzgesagt Video in 1200 Hours”}
{“prompt”:"",“completion”:“Milk. White Poison or Healthy Drink?”}
{“prompt”:"",“completion”:“How to Move the Sun: Stellar Engines”}
{“prompt”:"",“completion”:“Overpopulation & Africa”}
{“prompt”:"",“completion”:“An Antidote to Dissatisfaction”}
{“prompt”:"",“completion”:“1,000km Cable to the Stars - The Skyhook”}
{“prompt”:"",“completion”:“Neutron Stars – The Most Extreme Things that are not Black Holes”}
{“prompt”:"",“completion”:“What if We Nuke a City?”}
{“prompt”:"",“completion”:“The Billion Ant Mega Colony and the Biggest War on Earth”}
{“prompt”:"",“completion”:“What’s Hiding at the Most Solitary Place on Earth? The Deep Sea”}
{“prompt”:"",“completion”:“The World War of the Ants – The Army Ant”}
{“prompt”:"",“completion”:“Tiny Bombs in your Blood - The Complement System”}
{“prompt”:"",“completion”:“Could Your Phone Hurt You? Electromagnetic Pollution”}
{“prompt”:"",“completion”:“Is Meat Bad for You? Is Meat Unhealthy?”}
{“prompt”:"",“completion”:“Is the EU Democratic? Does Your Vote Matter?”}
{“prompt”:"",“completion”:“The Side Effects of Vaccines - How High is the Risk?”}
{“prompt”:"",“completion”:“The Most Dangerous Stuff in the Universe - Strange Stars Explained”}
{“prompt”:"",“completion”:“What If We Detonated All Nuclear Bombs at Once?”}
{“prompt”:"",“completion”:“The Origin of Consciousness – How Unaware Things Became Aware”}
{“prompt”:"",“completion”:“Loneliness”}
{“prompt”:"",“completion”:“Building a Marsbase is a Horrible Idea: Let’s do it!”}
{“prompt”:"",“completion”:“Is Organic Really Better? Healthy Food or Trendy Scam?”}
{“prompt”:"",“completion”:“Aliens under the Ice – Life on Rogue Planets”}
{“prompt”:"",“completion”:“How to Build a Dyson Sphere - The Ultimate Megastructure”}
{“prompt”:"",“completion”:“End of Space – Creating a Prison for Humanity”}
{“prompt”:"",“completion”:“The 12,019 Calendar IS HERE – A new calendar for humanity”}
{“prompt”:"",“completion”:“Why Beautiful Things Make us Happy – Beauty Explained”}
{“prompt”:"",“completion”:“Why Meat is the Best Worst Thing in the World”}
{“prompt”:"",“completion”:“How We Could Build a Moon Base TODAY – Space Colonization 1”}
{“prompt”:"",“completion”:“Wormholes Explained – Breaking Spacetime”}
{“prompt”:"",“completion”:“What If You Detonated a Nuclear Bomb In The Marianas Trench?”}
{“prompt”:"",“completion”:“Plastic Pollution: How Humans are Turning the World into Plastic”}
{“prompt”:"",“completion”:“3 Arguments Why Marijuana Should Stay Illegal Reviewed”}
{“prompt”:"",“completion”:“The Deadliest Being on Planet Earth – The Bacteriophage”}
{“prompt”:"",“completion”:“The Black Hole Bomb and Black Hole Civilizations”}
{“prompt”:"",“completion”:“Time: The History & Future of Everything – Remastered”}
{“prompt”:"",“completion”:“A Selfish Argument for Making the World a Better Place – Egoistic Altruism”}
{“prompt”:"",“completion”:“String Theory Explained – What is The True Nature of Reality?”}
{“prompt”:"",“completion”:“Homeopathy Explained – Gentle Healing or Reckless Fraud?”}
{“prompt”:"",“completion”:“Why Alien Life Would be our Doom - The Great Filter”}
{“prompt”:"",“completion”:“How to Make an Elephant Explode”}
{“prompt”:"",“completion”:“Universal Basic Income Explained – Free Money for Everybody? UBI”}
{“prompt”:"",“completion”:“Emergence – How Stupid Things Become Smart Together”}
{“prompt”:"",“completion”:“How to Cure Aging – During Your Lifetime?”}
{“prompt”:"",“completion”:“Why Age? Should We End Aging Forever?”}
{“prompt”:"",“completion”:“How Bacteria Rule Over Your Body – The Microbiome”}
{“prompt”:"",“completion”:“Is Reality Real? The Simulation Argument”}
{“prompt”:"",“completion”:“What Happens If We Bring the Sun to Earth?”}
{“prompt”:"",“completion”:“Why Black Holes Could Delete The Universe – The Information Paradox”}
{“prompt”:"",“completion”:“What Happens If We Throw an Elephant From a Skyscraper?”}
{“prompt”:"",“completion”:“Optimistic Nihilism”}
{“prompt”:"",“completion”:“The Rise of the Machines – Why Automation is Different this Time”}
{“prompt”:"",“completion”:“The Last Light Before Eternal Darkness – White Dwarfs & Black Dwarfs”}
{“prompt”:"",“completion”:“Are GMOs Good or Bad? Genetic Engineering & Our Food”}
{“prompt”:"",“completion”:“Do Robots Deserve Rights? What if Machines Become Conscious?”}
{“prompt”:"",“completion”:“Why Earth Is A Prison and How To Escape It”}
{“prompt”:"",“completion”:“Overpopulation – The Human Explosion Explained”}
{“prompt”:"",“completion”:“A New History for Humanity – The Human Era”}
{“prompt”:"",“completion”:“The Most Gruesome Parasites – Neglected Tropical Diseases – NTDs”}
{“prompt”:"",“completion”:“Fusion Power Explained – Future or Failure”}
{“prompt”:"",“completion”:“The Most Efficient Way to Destroy the Universe – False Vacuum”}
{“prompt”:"",“completion”:“Genetic Engineering and Diseases – Gene Drive & Malaria”}
{“prompt”:"",“completion”:“Genetic Engineering Will Change Everything Forever – CRISPR”}
{“prompt”:"",“completion”:“Death From Space — Gamma-Ray Bursts Explained”}
{“prompt”:"",“completion”:“What Happened Before History? Human Origins”}
{“prompt”:"",“completion”:“What Are You?”}
{“prompt”:"",“completion”:“The Fermi Paradox — Where Are All The Aliens?”}
{“prompt”:"",“completion”:“How Small Is An Atom? Spoiler: Very Small.”}

Is 101 titles too little data?
Should I remove ones that aren’t contributing much like “What are you” and “Lineliness”?

Hyperparameters:

I’m using the following hyperparameters, but I probably should use some more and should change the values of these too maybe:

–model curie --batch_size 4 --no_packing

If you have any suggestions, please explain the reasoning behind it to help me understand, thank you :slight_smile:

You would use our CLI data preparation tool which gives you suggestions and justifies them.

openai tools fune_tunes.prepare_data - f your_file.jsonl

Do you manage Kurzgesagt channel?

Okay, thank you.

No, I don’t, although I work with different YouTubers, and I’m testing if GPT-3 might help them.

1 Like

Make sure you understand the legal implications on training on somebody else’s data. I don’t know what they are but you may want to consult with a lawyer

2 Likes

Okay, thank you for the heads up.