Pro Tips: Tokenizer

Understanding the BPE and Tokens/Tokenizer is extremely helpful as you advance in your prompt designs and think about advanced applications.

Strongly suggest reading up and playing with the Tokenizer:
https://beta.openai.com/tokenizer?view=bpe

I go into some depth on why this is important in this GPT3 101 essay/tutorial On Structure:
https://un1crom.medium.com/gpt3-linguistics-101-a-multi-part-series-this-is-part-1-on-structure-a41af3a77353

9 Likes

Been reading Part 1 & Part 2 and they have been really useful in increasing my understanding of GPT-3 in particular and LMs in general.

Looking forward to reading Part 3 when it comes out.

In the Part 2, there is a link to the slides you used at your community talk. I was wondering if there is a recorded version of the talk posted somewhere.

Also, if you recommend any further material on prompt designs for gpt-3.

1 Like

Parts 3.1-3.3 are up. You’ll find em :slight_smile:

Let me find the community talk. It’s somewhere out there.

Found the Parts 3.x :slight_smile: