3

I’d like to know too, I’m uploading in HTML and wonder if its being distracted with all the metadata for the DOM. I’m thinking of just rewriting the important stuff in JSON

4

I am very interested about this question also! I suppose it depends how the Knowledge module works behind the scene. I have a doubt that it only works as pure extra “textual” instructions. So I would say txt…

5

Nope, each document type has different characteristics.

A plain text file will be parsed very fast and a .XLSX file very slow.

For .XLSX a code interpreter is needed and for .HTM not.

And .XLSX can trigger “structured data” which can be “smart interpreted” by the GPT, where a .PDF won’t do.

So it really depends on the semantics, taxonomy and context of your purpose.

It’s not “just” a filetype, since the filetype powers the knowledge.

2 Likes
6

I have been testing with about 250.000 high-end dedicated articles about one specific subject.

I tested those formats;

  • JSON
  • XLSX
  • PDF
  • HTM
  • TXT

Uploaded as 750mb textfile and it didn’t make any sense for complex questions.

Uploaded as 150mb Excelsheet was superior.

It was able to read the data, fetch the columns, parse data and date, linked it’s knowledge to one fact and the other, etc…

But is was slowwwwwwwwwwwwwwwwww, so I decided to host the SQL myself and wrote a simple API with an endpoint to my server.

You get what you feed it; structured data in = structured data out and plain text in = plain text out.

1 Like
7

Must be using a low speed library to open MS excel files, and running that code for every call.

8
  1. Results : impressive
  2. Speed : not impressive
9

presumably, a CSV exported version would be fast.

10

Yes,

Yes, that worked and was fast - but limited in term of structured data and the way GPT did associate rows / cells (comma separated).

The Excel was able to act as a wrapper for 250.000 articles, the .CVS ended by something like 8.000 (GPT complained the context / content was too much).

So it works, when you have “little” data only (couple of hundreds of articles, not tens of thousands).

11

If I have some PDFs that contain academic papers and I’d want to train the custom GPT with that knowledge, would you recommend excel? So far with the PDFs, it didn’t make any sense for complex questions, like you said.

12

Depends on the structure of your date.

In my case the source of all knowledge was a SQL database with fields like “date / location / item / subject / source / etc…”.

This way GPT was able to “link” certain topics (named in the “subject” row) with “dates” and “locations”, giving interesting results.

But when it’s “just” an article XLS doesn’t make much sense.

GPT tries to find the date and location for an article itself, e.g. when this is the format;

NEW YORK, 2023/11/12 - And some interesting line of text to fill up this fake item…

This way GPT tries to see the logic {CITY} / {DATE} - {CONTENT} by itself.

Just try out what works best for you; I uploaded many, many files and a GB of data… and deleted them al with the click of one mouse button (because XLS was giving the best results, but in the slowest possible way).

1 Like
13

I am noticing more hallucinating / made up data when providing a data source that is a csv. It seems to stick to the provided knowledge files when I share a txt doc.

In both cases, have the same instructions to not make up answers and stick to the provided data source.

Curious if anyone else has seen something similar.

14

For (unstructured) retrieval I am thinking markdown would be best. PDFs are a big “meh”. They can be unreliable and hard to read. It would be worth running the PDF thru Code Interpreter and seeing how easy it is to read first. Last I checked they were using a very basic parser

Would love to see some tests.

For structured formats like CSV & JSON I believe it would be better to use an API (Actions). I know we’re strictly discussing GPTs but for API I found GPT really good at creating GraphQL queries.

15

I have uploaded multiple PDF, but the hit rate is super low. And it is even worse than my personal embedding experiment (very simple vector database usage and similarity search). So I wondering how GPT is doing file processing.

16

Markdown syntax seems to be ideal for this type of thing because it gives semantic meaning to the text (headings, callouts, quotes, emphasis, etc) without all the clutter from HTML. It’s also what powers the text rendering behind chatGPT. What are your thoughts on Markdown? Anyone done testing on it?

As for hallucinations, make sure you add in the system message to “not make it up if the answer can’t be found in the source material”. Be sure to also mention to “think step by step”. This has virtually eliminated hallucinations for me.

4 Likes
17

Agreed, I am much happier with the results when using Markdown. I also pay close attention to the code interpretor log at the end of each reply, it suggests that the amount of data actually analysed seems to always be between 500 and 2000 characters, so I make sure I keep my knowledge files below that.

18

And I forgot to say that I always ask ChatGPT to review, validate and reformat my Markdown to optimise future analysis

19

To add to this convo, and I wish we had an OpenAI employee knowledgable enough to assist and chime in, I am curious if when using a .zip, what the added latency is, and if once uncompressed, does it remain so for future use, with the files stored somewhere? Curious on the long-term GPT storage mechanics, and yes if the .zip only needs a single first-use unzip.

Might add, is there a file or .zip size limit anyone knows of?

20

Prespan: Usually, I speak and write German and not English which will probably be reflected in my usage of the English language.

I tested the following formats:

  • PDF
  • XLSX
  • DOCX
  • RTF
  • XML
  • MD

In my experience, Markdown (MD) works best for structured text that also uses formal symbols such as in math equations. The - conjectured, admittedly - reason for this is due to the similarity of the data format used for the output and the availability of (structured) training data in that format.

Whenever I ask for math assistance, Markdown with embedded or MathJAx‘d equations is used to render the output on the client side.

The syntax is also used to display e. g. tables, headings in various hierarchical depths and embed graphics.

On the input side, the methods used to preprocess data is not known to me with certainty. Thus the addendum from above that the reasons are a conjecture.

The problem at the present is that there is little functionality built in the system to transform the knowledge in - say PDF - to formats that produce higher quality output in a faster way.

For the example under consideration, and small knowledge data, a service called MathPIX (free version) was employed for the conversion of PDF with math expressions to MD and then manually dumping the result of the preprocessing step outside the AI system in the relevant input fields worked best but - to be blunt - feels clumsy and inconvenient.

The idea originates from another problem that still awaits its solution, namely the handling of multiple PDFs. This worked without system failure only with a low probability in ~ 30 tries. With markdown as the only input format, the problem was absent in the case of knowledge distributed among multiple files.

To sum up, I distilled the following „guidelines“ for my purposes:

  • Use Markdown in the multiple file case.
  • Use Markdown in the case of a single file that contains elements in languages other than the natural one.

Ideally, a conversion would be possible directly in the system or an integration to outside services is possible. Furthermore, I am a bit skeptical that the product fulfills the data protection criteria because - at the present - it is possible to download knowledge using prompts or the code interpreter, if activated.

Best wishes
David

2 Likes
21

Don’t forget to check the “Code Interpreter” checkbox.
If you don’t, your GPT will probably work, but I’m not sure if it’s really checking the knowledge files.

I solved most of my issues (I’m using CSV file format)

22

GPT can see the knowledge with code interpreter disabled.