# Embedding models with an array of integers input **URL:** \ **Category:** API\ **Tags:** api\ **Created:** [August 9, 2024, 1:18am UTC](https://community.openai.com/t/embedding-models-with-an-array-of-integers-input/900165 "2024-08-09T01:18:33Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![gshy2014](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/gshy2014/32/336304_2.png) [@gshy2014](https://community.openai.com/u/gshy2014)\ **Post date:** [August 9, 2024, 1:18am UTC](https://community.openai.com/t/embedding-models-with-an-array-of-integers-input/900165/1 "2024-08-09T01:18:33Z")
I notice that openai embedding API can take an array of integers or an array of arrays of integers as input (Cf: [https://platform.openai.com/docs/api-reference/embeddings/create#embeddings-create-input](https://platform.openai.com/docs/api-reference/embeddings/create#embeddings-create-input)). Are there any assumptions about how this array of integers are generated? Otherwise, how does it make sense to embedding a bunch of indices without knowing what they are? ---
**Author:** ![vasyl](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/vasyl/32/274373_2.png) [@vasyl](https://community.openai.com/u/vasyl)\ **Post date:** [August 9, 2024, 2:58am UTC](https://community.openai.com/t/embedding-models-with-an-array-of-integers-input/900165/2 "2024-08-09T02:58:26Z")
Hi @gshy2014 , I’ve checked the url and didn’t find anything similar to what you’re saying. I think you’d have to elaborate on what you’re looking for and what’s exactly that you can not find or understand. It seems you might have a misconception about how embedding models function. These models process information and generate an embedding vector— **essentially an array of numbers** —that captures the underlying meaning and relationships of words, numbers, images, and other data types. ---
**Author:** ![Diet](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/diet/32/562749_2.png) [@Diet](https://community.openai.com/u/Diet)\ **Post date:** [August 9, 2024, 5:03am UTC](https://community.openai.com/t/embedding-models-with-an-array-of-integers-input/900165/3 "2024-08-09T05:03:56Z")
Welcome to the community! > [@gshy2014](#): > > Are there any assumptions about how this array of integers are generated? Otherwise, how does it make sense to embedding a bunch of indices without knowing what they are? I think you’re talking about this ![image](https://us1.discourse-cdn.com/openai1/original/4X/5/e/0/5e075cd55a48f976aabf951d24d5249b56a67c79.png) The integers here are actually token IDs. You get these token ids by using the correct tokenizer model ([GitHub - openai/tiktoken: tiktoken is a fast BPE tokeniser for use with OpenAI's models.](https://github.com/openai/tiktoken)). Most devs just use the tokenizer to calculate costs, but OpenAI uses the tokenizer as the first stage in their models (it’s probably a one hot encoding) before getting to the actual meat of the transformer. You can theoretically do the tokenization step yourself and pass the results as an array, but you, as a developer, gain pretty much nothing from that Theoretically potentially the JSON payload might end up being smaller if you want to save bandwidth because it might compress better. But realistically probably not really. It’s a cool find, the practical uses are probably pretty niche. ---
**Author:** ![gshy2014](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/gshy2014/32/336304_2.png) [@gshy2014](https://community.openai.com/u/gshy2014)\ **Post date:** [August 10, 2024, 7:57pm UTC](https://community.openai.com/t/embedding-models-with-an-array-of-integers-input/900165/4 "2024-08-10T19:57:39Z")
> [@Diet](#): > > GitHub - openai/tiktoken: tiktoken is a fast BPE tokeniser for use with OpenAI’s models. Thanks for the explanation. It makes sense. Is the use of integer array (ids from tiktoken) as input to the API documented anywhere? Or is it only intended for internal use ?