From reading various docs, the GPT LLMs were trained on tokens (sub-words) using an initial embedding vector as the first layer of its internal neural network. That is, there’s an embedding vector for each token. But in a RAG architecture, the OpenAI embeddings API returns an embedding for a strin…

Embedding tokens vs embedding strings?

ginaaaaaaaaaaa February 9, 2024, 12:32pm 8

Hi, I am still not quite sure i understood correctly - will I get an embedding vecotr per token or per sequence ? I have tried using the text-embedding-ada-002… but could not figure it out

Topic		Replies	Views
Embedding - Usage for all GPT text to number applications? Documentation embedding	6	425	October 13, 2024
How can I send vectors as a chat context? Prompting embeddings	8	8681	May 15, 2023
Can someone make embeddings make sense? (Not what you think, more in post, lets discuss!) API embeddings , gpt-4	6	2251	September 19, 2023
Embeddings as model input API embeddings , api , prompt	3	2435	June 16, 2023
Does openAI provide API that takes Embeddings as an input? API embeddings	10	4139	December 18, 2023

Embedding tokens vs embedding strings?

Related topics