Embedding - text length vs accuracy?

API
7

Hi @mattcorbin

Its easy if you store both the text (what you call “chunks”) and the embedding vectors in DB table rows.

Then, when you do a query you can select the search method based on the search text (length, context, etc)

HTH

:slight_smile:

Appendix: Example DB schema (FYI, only)

 create_table "embeddings", force: :cascade do |t|
    t.string "model"
    t.string "text"
    t.string "vector"
    t.datetime "created_at", precision: 6, null: false
    t.datetime "updated_at", precision: 6, null: false
  end
1 Like