# Preprocessing for embeddings **URL:** \ **Category:** API\ **Created:** [July 11, 2023, 3:31pm UTC](https://community.openai.com/t/preprocessing-for-embeddings/295017 "2023-07-11T15:31:08Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![yemane](https://avatars.discourse-cdn.com/v4/letter/y/f08c70/32.png) [@yemane](https://community.openai.com/u/yemane)\ **Post date:** [July 11, 2023, 3:31pm UTC](https://community.openai.com/t/preprocessing-for-embeddings/295017/1 "2023-07-11T15:31:08Z")
I’m currently trying to do some topic modeling on articles. I have many (40+) possible categories. I’m currently doing something similar to [Recommendation\_using\_embeddings.ipynb](https://github.com/openai/openai-cookbook/blob/main/examples/Recommendation_using_embeddings.ipynb). The tutorial in the link doesn’t include any pre-processing of the text sent to the `text-embedding-ada-002` model. I am wondering if pre-processing the text of the article makes sense. Like removing stop words and punctuation, making words lowercase, etc… Since I’ve found an article with someone using another embeddings model and the person did do pre-processing [multi-class-text-classification-with-doc2vec-logistic-regression](https://towardsdatascience.com/multi-class-text-classification-with-doc2vec-logistic-regression-9da9947b43f4). If you agree pre-processing makes sense. Is there a best practice for what kind of pre-processing to do? ---
**Author:** ![AidanM](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/aidanm/32/146909_2.png) [@AidanM](https://community.openai.com/u/AidanM)\ **Post date:** [July 11, 2023, 3:34pm UTC](https://community.openai.com/t/preprocessing-for-embeddings/295017/2 "2023-07-11T15:34:54Z")
I’ve never thought about pre-processing, so it might be a good idea. For my own personal purposes, without the pre-processing, I have found lots of success with embeddings, so I think you are fine. I use embeddings for text that gets lifted off of construction documents, so there is little consistency with punctuation and capitalization. ---
**Author:** ![wfhbrian](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/wfhbrian/32/19388_2.png) [@wfhbrian](https://community.openai.com/u/wfhbrian)\ **Post date:** [July 11, 2023, 3:39pm UTC](https://community.openai.com/t/preprocessing-for-embeddings/295017/3 "2023-07-11T15:39:36Z")
Pre-processing the text before using it with the `text-embedding-ada-002` model can indeed be beneficial, but isn’t necessary. A much less sophisticated model is used in the “multi-class-text-classification-with-doc2vec-logistic-regression” example. Less sophisticated models benefit more from the reduction-type preprocessing strategies you listed. For `text-embedding-ada-002`, the type of pre-processing I would consider would be more likely to be _additive_, as they would add content to the embedding text input, rather than remove it. This would increase the overall relevant context provided to the model. Anything that might be necessary for your specific solution. The specific pre-processing steps you choose will depend on your specific use case, the characteristics of your text data, and the model you choose for embedding. It can be helpful to experiment with different pre-processing techniques and evaluate their impact on the quality of the embeddings and the performance for your desired outcome. I hope this helps! Let me know if you have any further questions. ---
**Author:** ![yemane](https://avatars.discourse-cdn.com/v4/letter/y/f08c70/32.png) [@yemane](https://community.openai.com/u/yemane)\ **Post date:** [July 26, 2023, 7:04pm UTC](https://community.openai.com/t/preprocessing-for-embeddings/295017/4 "2023-07-26T19:04:38Z")
The use case is trying to use embeddings to find the most similar topic to an article. A kind of topic modeling/classification. Would it make sense to make everything lowercase? And remove non-alpha-numeric characters in this case? Thinking these steps would at least remove some unnecessary noise ---
**Author:** ![EricGT](https://sea2.discourse-cdn.com/openai1/user_avatar/community.openai.com/ericgt/32/20571_2.png) [@EricGT](https://community.openai.com/u/EricGT)\ **Post date:** [December 17, 2023, 5:01pm UTC](https://community.openai.com/t/preprocessing-for-embeddings/295017/5 "2023-12-17T17:01:12Z")