Efficient Methods for Processing Large Volumes of Tabular Data with ChatGPT

Prompting
1

Hello everyone,

I am currently working with large volumes of tabular data and would like to leverage ChatGPT for processing and analyzing this data. I am particularly interested in understanding the most efficient methods for handling such extensive datasets using ChatGPT.

Could you please share any best practices, techniques, or tools that might enhance the performance and accuracy when working with large-scale tables in ChatGPT? Additionally, if there are any specific challenges I should be aware of when processing large datasets, I would appreciate any insights on how to address them.

Thank you in advance for your guidance!

2

Hi,

LLM’s and large blocks of numeric data do not play great together.

You can for sure get insights form tabular data in reasonably small sizes, but for large datasets current gen LLM’s are not ideal.

The sorts of things that work well are say 10 departments sales figures over a 12 month period in monthly segments.

It may even work well with hundreds of departments, I’ve not tried it.

LLMS of this type rely on attention being used to find what’s important in the input data, a big string of numbers is difficult to process like this.