Does anyone have experience with the text-embedding-ada-002 model? The blog post is kinda vague: [image] New and improved embedding model We are excited to announce a new embedding model which is significantly more capable, cost effective, and simpler to use. …

Excellent! I know a doctor who just started a project that requires some light meta-analysis of radiology curricula in the context of family practice residency programs. I’m thinking this might help!

Hi, if you wouldn’t mind clarifying a bit: Do I use text-similarity to perform semantic search on a document? I don’t have the research papers yet, so to practice, I want to summarize Lewis Carroll’s corpus and be able to query it about Wonderland. If my understanding is correct, a query like “Who …

text-similarity is one of the old models. Now you need to use text-embedding-ada-002 instead. I think the old models still work though

Ah! I need to read more closely. This was staring me in the face: openai/openai-cookbook/blob/main/examples/Semantic_text_search_using_embeddings.ipynb { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## Semantic text search using embeddings\n", …

Yes, it doesnt have to be a csv file. It could be a tab delimited file, or a database, or any other data source The examples use a CSV file for simplicity. You can also view the vectors in the file once you have done the embedding. The food review database is opensource and, from memory, the url ha…

import openai import numpy as np import math import pandas as pd with open("AAiW/alice_in_wonderland.txt", "r") as f: data = f.read() # Calculate the number of tokens in each chunk chunk_size = math.ceil(len(data.split()) / 5) # Split the text into chunks chunks = [data[i:i+chunk_size] for i …

[image] Dent: Interesting! So are its use cases closer to semantic search? Yes, look at how the model (text-embedding-ada-002) perceives these comparative tests using the OpenAI API embeddings and a hand-coded cosine similarity function (in Ruby): irb(main):023:0> Embeddings.test_strings("do…

The embeddings are not free - but they are very very cheap

Thanks. Good catch @raymonddavey … Edited my post: Running these tests (using the OpenAI API embedded method) is currently free, very cheap …

Thanks for reviving this thread! The person I was helping with this has been incommunicado, but it’s great to have this info for myself and other users. Does ruby-openai not include embeddings_utils? Bc the Python version has cosine_similarity

`text-embedding-ada-002`

API

ruby_coder January 24, 2023, 4:47am 23

More discussion and testing here:

Topic		Replies	Views
Question on text-embedding-ada-002 API	11	6652	July 27, 2023
Some questions about text-embedding-ada-002’s embedding API	145	47230	November 24, 2023
Which database tools suit for storing embeddings generated by the Embedding endpoint? API	45	27289	September 22, 2023
Semantic search using uploaded files (only performs lexical search for me) API	18	2853	December 11, 2022
Would using embeddings like this work in theory? API python	5	2626	December 20, 2023

`text-embedding-ada-002`

Related topics