# Some questions about text-embedding-ada-002’s embedding

**URL:** <https://community.openai.com/t/some-questions-about-text-embedding-ada-002-s-embedding/35299>\
**Category:** API\
**Created:** [January 13, 2023, 9:29am UTC](https://community.openai.com/t/some-questions-about-text-embedding-ada-002-s-embedding/35299 "2023-01-13T09:29:32Z")\
**Posts on this page:** 1\
**Showing post:** 77

<div class="post-metadata">

**Author:** ![curt.kennedy](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/curt.kennedy/32/709249_2.png) [@curt.kennedy](https://community.openai.com/u/curt.kennedy)\
**Post date:** [October 19, 2023, 6:21pm UTC](https://community.openai.com/t/some-questions-about-text-embedding-ada-002-s-embedding/35299/77 "2023-10-19T18:21:33Z")

</div>

> [@\_j](#):
>
> Or a real question behind my rambling: how is the language model of embeddings _fine tuned_, and how does that affect weights, vectors, embeddings?

No idea if internal embeddings are changed with a fine-tune. I just assumed the neural weights changed. The main reasoning is that the semantics, once trained, shouldn’t change, and the fine-tune just reshapes the output from the input (unchanged) semantics.

> [@\_j](#):
>
> For a codex embedding, does one just train it more on code, and then it is able to distinguish more sequence semantics.

Yes the codex is primarily trained on code. But rolling everything into one new model, and deprecating the rest, suggest they went with a Mixture of Experts (MoE) thing similar to GPT-4 (rumors, I know), and this consolidation would basically get the new models all on the same architecture, possible saving some money in the process.

> [@\_j](#):
>
> One might extract 50k single-token embeddings for posterity before these models go away.

Go for it!

---

_[View the full topic](https://community.openai.com/t/some-questions-about-text-embedding-ada-002-s-embedding/35299)._