IM using gpt4omini 128k Token Limit Model.
Assuming I’m passing a content that hits 100k tokens. [ 50k tokens of article content + 35k tokens of style guide (how to write articles guide) + 2k tokens of prompt ]
Style-guide: Welcome - Microsoft Writing Style Guide | Microsoft Learn
Outcome : to check for spelling and grammar mistakes based on a predefined style guide.
I have used response_format parameter by defining json schema structure.
My question is:
-
Is the Json schema in response_format parameter also considered additionally as tokens, apart from the 100k passed.
-
while passing 50k tokens of articles, how good is 4o-mini in remembering the additional- 35k tokens of style guide points and then again follow up with prompt of 2k. How will the accuracy look like?
Based on my experience in RAG, after Top 10 chunks, the ability of OpenAI to craft response slows down. Will the same case happened in this situation?