…[SNIP]… …[SNIP]…GPT-Neo outperformed GPT-3 Ada on Hellaswag and Piqa. Hellaswag is an intelligent multi-choice sentence completion benchmark that has a context paragraph and four endings. Piqa measures common sense reasoning where the machine has to pick one out of two sentences that make the most sense. GPT-Neo also outperformed GPT-3 Ada on Winogrande, a benchmark that uses common sense to resolve ambiguous pronouns in a sentence. However GPT-3 Davinci, the largest version of GPT-3, with about 65 times as many parameters, comfortably beats GPT-Neo in all the benchmarks, as you would expect. [Source]
2 Likes
“More data = good.” ~ GPT-3 (and me)
2 Likes
Screenshot of NovelAI frontend …It sounds like they’re seeing success with a smaller model better fine-tuned with this task in mind…
