Is there a way to update a CSV as potential input for a Chat Completion? I see that there is a file update API (but apparently only for fine tuning) but the chat completion API doesn’t mention files. GTP 4 told me this: Unfortunately, it is not possible to give the ChatGPT API access to local fil…

Hi @oliver.drobnik Welcome to the community. This is indeed hallucination/confabulation. [image] Using my own knowledge base with GPT-4 Community Welcome to the OpenAI Community. When it comes to answering questions from large knowledge bases. Embeddings + Completions…

Ok, but what about my first question. Is there a way to facilitate this flow of data?

If you read the docs, you’ll understand that you don’t need to upload the csv, instead you break it into chunks and obtain embeddings. Here’s the boilerplate code given in docs: def get_embedding(text, model="text-embedding-ada-002"): text = text.replace("\n", " ") return openai.Embedding.cr…

Hello @oliver.drobnik Yes you can, and it works very well. Here are the steps you can take. Parse the CSV file and for every row, modify the row to one line of key value sentence. For example Name, Product, Review Oliver, SD Card, I think this SD Card is great Nelson, SD Card, This SD Card is…

Please forgive my ignorance. Also, I’d like to understand the steps at the lowest level using only the JSON-REST API. So it seems to me that I should be able to change my CSV into embeddings. That’s done via an OpenAI embedding model, right? like https://platform.openai.com/docs/guides/embeddings/w…

Why use embedding The reason you want to use an embedding is because you want to find the most relevant context before you feed into your prompt, and also to manage prompt tokens. If this is not what you need, then you probably don’t need to use embedding. What data to encode into embedding You n…

thank you for the extensive reply. Gives me a lot to think about for my specific use case. Basically the core message is: I need to provide sufficient relevant facts together with my query for GPT to do its magic. If it is about text then embeddings and vector databases seem to be what most people …

[image] nelson: Most Vector databases will only manage the embeddings for you, so when you perform a semantic search, most likely you will get back an ID and the embeddings with scores. You will need to map this ID to another database to retrieve the actual text. I generally avoid this as it …

Thanks for the added input Bill. Cheers!

Hi Oliver, You are correct, most of the techniques are using embeddings and vector database to do searches. Another important technique is the art of prompt engineering, if you can find creative ways to use this into your workflow, it will make a lot of difference. Wish you the best of luck in you…

Send CSV file for use in Chat Completion?

API

sps May 8, 2023, 4:47pm 2

Hi @oliver.drobnik

Welcome to the community.

This is indeed hallucination/confabulation.

Topic		Replies	Views
How can I send vectors as a chat context? Prompting embeddings	8	9640	May 15, 2023
Reducing Cost of GPT 4 by using embeddings Prompting	23	11076	May 4, 2023
About the usage of ChatGPT Embedding API	9	4784	August 18, 2023
Using GPT to Search & Pull Recommendations from a Database? API	23	11131	August 22, 2024
How to feed data for completions, instead of using prompt/answer fine-tuning format? API	25	18430	December 17, 2023

Send CSV file for use in Chat Completion?

Related topics