Understanding the BPE and Tokens/Tokenizer is extremely helpful as you advance in your prompt designs and think about advanced applications.
Strongly suggest reading up and playing with the Tokenizer:
https://beta.openai.com/tokenizer?view=bpe
I go into some depth on why this is important in this GPT3 101 essay/tutorial On Structure:
https://un1crom.medium.com/gpt3-linguistics-101-a-multi-part-series-this-is-part-1-on-structure-a41af3a77353
9 Likes
Harish
2
Been reading Part 1 & Part 2 and they have been really useful in increasing my understanding of GPT-3 in particular and LMs in general.
Looking forward to reading Part 3 when it comes out.
In the Part 2, there is a link to the slides you used at your community talk. I was wondering if there is a recorded version of the talk posted somewhere.
Also, if you recommend any further material on prompt designs for gpt-3.
1 Like
Parts 3.1-3.3 are up. You’ll find em 
Let me find the community talk. It’s somewhere out there.
