Large JSON Responses from Assistant API are truncated

[IntelliJJ]

As far as I know GPT-4o still has a 8k token limit, so yes this is expected.

You might find this thread interesting:

Using gpt-4 API to Semantically Chunk Documents

I am creating this topic based upon the thread of conversations which developed from this post: RAG is not really a solution - #43 by SomebodySysop While a number of us have different approaches to the first tier of “Semantic Chunking”, like @sergeliatko 's API: RAG is not really a solution - #50 by sergeliatko we were all still looking for a way to have the LLM do that last bit of “sub-chunking”, where instead of simply chopping the chunk up by size we have the model look at it and determine t…

TL;DR: See if you can prepend line numbers and have the model return start and end line numbers instead of full text.