Generate Embedding For each Tabular record

Prompting
1

Hi All,

I am trying to generate embedding for each tabular records, i have listed down 2 approaches(attached),

openai-Page-2.drawio
openai-Page-2.drawio951×141 3.66 KB

please help/guide which approach could generate rich embeddings, or point to other better way
Thanks
Atul loona

2

Skip the generating and just do something like this:

Column name: row value

For example, if you have columns Name and Date:

Name: Brian
Date: 2023-03-20
3

Thanks

4

Thanks

Solution

5

Dear wfhbrian, Community Members

  1. I tried to generate embeddings by combining columns values into 1 column, it did generate embeddings.

  2. Table has both string and numerical values, i tried with only string columns and using both(string and numerical columns)

  3. here are observations and results
    3.1-> I did regression analysis using embedding with only string columns: mse=0.58 - better
    3.2-> I did regression analysis using embedding with string and numerical columns : mse = 0.60

So coming back to original question, i have to include numerical values in my dataset, and somehow represented or embedded in embedding vector.

So original thought was to generate all string paragraph or generate summary for better embeddings.

please guide how to get better embedding with numerical data.

Regards
Atul loona

1 Like