I m looking to pass the 10-20k characters of JSON data to openai and ask questions about it but i m running into issues with max tokens, is this something you could help me with? (Python)
All models have a finite context length (max tokens) which cannot be exceeded.
Your specific use-case requires embeddings. Here’s the official OpenAI docs on embeddings
1 Like