Hi, are the token embeddings learned together with the other weights in the codex (or indeed in the gpt3) model? I couldn’t find a description of how the embeddings arise in the codex/gpt papers, but i may have missed it in the references or something. Any links to further info on this would be much appreciated. Thanks.
Related topics
| Topic | Replies | Views | Activity | |
|---|---|---|---|---|
| Embeddings for tokens used by GPT models? | 2 | 824 | December 17, 2023 | |
| Methodological information about embeddings | 3 | 485 | January 3, 2024 | |
| Embedding tokens vs embedding strings? | 12 | 6866 | February 11, 2024 | |
| Passing token weights to embeddings API? | 5 | 1549 | November 14, 2023 | |
| Embedding vector and weights/bias relationship? | 0 | 52 | August 4, 2024 |