12

Hi Drew,
It’s a combination of reading unstructured data from documents using different downstream tasks, creating a database of embeddings and using the OpenAI Completion API. Feel free to message me if you are interested in testing it out.
Cheers

13

Hi Nelson, I tried it out and gave you some feedback via linkedin message. I am wondering if I missed a step though, since it didn’t seem to me that the app was relying on my library. Is one’s library ready almost instantaneously? Mine appeared to be but that seems unrealistic if embeddings are being created from the text.

14

@nelson - Can you provide me pointers to your code ? I am looking for something similar

2 Likes
15

@vasanth.finance
Sure are you looking to write code to build your self from scratch or using integration like Google Spreadsheets with Open API?

1 Like
16

Found your message! That makes sense. One of the hardest part of using large language model is to fact check against the answer. I’ll take deeper on your use case and see how to we provide better results, stay tuned.

1 Like
17

Hi Nelson,

great approach. Really interested, maybe you could reach out to philipp.hoellermann@dwg.de. We are thinking about “fine-tuning” the system with a lot of study materials (lots of it in PDF format) to create an online learning tutor. Do you think your solution might work into this direction?

Best, Philipp

1 Like
18

Hi Philipp

We could test it out and see if it will work for you use cases. Let’s chat.
Cheers
Nelson

1 Like
19

Any github code? This sounds amazing!

2 Likes
20

Is there a link so we can test ride?

21

Hi Raul,
Since it is fairly new and not completely polish, I’m inviting one person at a time and guide them thru the on boarding process. You do need to have an active OpenAI account setup to use it. When you ready please message me, and I will walked you thru the process. Thanks.

22

How are you using the Completions and embeddings together? I can’t see a way in the api to pass the embeddings to the completions endpoint? None of openai’s examples show anything similar. At the moment I’m only able to pass one page worth of content from my site via the prompt to answer questions about that specific page (before I run out of tokens due to input length), I would love to be able to do my entire site in an easier way so would love to hear how you’re doing it!

1 Like
23

Hi Dale,

That makes sense, the completions API only accept text and return text, the Embedding API accepts text and return most relevant embedding. You can look at this thread I have Google Spreadsheet + GPT3 - #28 by notifications, the gsheet in there demonstrate how to do this and the source code is also embedded in the App Script.

24

What you need to do is compare the user input embedding (done on the fly) to your database of embeddings using cosine similarity. Based on similarity scores, select the most relevant embeddings. Then go back and retrieve the original text corresponding to that embedding, and add that text to your prompt.

2 Likes
25

Hello! This looks super cool and similar to what I’m trying to do:

I’m trying to generate documentation based off our code and have it make updates as our code changes (without having to update the docs manually)

The thing that I’m struggling with is the required input/output. What do you enter as the output for your PDFs?

Thanks!

2 Likes
26

Hi Peter,

Auto generate documentation from code is very cool, I’ll love to use it myself. Also if there is way to simply ask what the code does will be very cool as well. Which Open AI model are you using to generate the documents?

Regarding to the input and output for the app, it’s all text based.

1 Like
27

Thanks! I’m following the OpenAI docs and sticking to davinci 3 for now until I get it right.

What I’m unsure about is how I “ingest” the data without having to also submit the results like in the examples in the docs.

I think what I need to do is use the Files api to upload all the functions, upload the current state of the docs and ask “which functions need updating” but that’s been the hard part.

Thanks!

28

Hi Peter,
I was just wondering, have you tried different models without fine-tuning?
It’s possible that Fine Tuning might not be necessary, what are some of the results you are getting back on different standard models?

1 Like
29

Hey, sorry for the delay! I have tried it and yes you’re right it works great.

I’m thinking about using this to update changes that are happening on a regular, daily basis. For example, being able to update the docs based off an API change in a pull request.

I’ve been messing around with LangChain recently and created this in the process: https://react-gpt.fly.dev

30

Very cool Peter. Glad that it worked out for you!

31

Hi all, the app is now live and it is free to try at https://superinsight.ai
Cheers!