Large context window — what are you using it for?

Community
1

Was just reminded that GPT-4.1 has the largest context window of all the OpenAI models — up to 1,047,576 tokens. Has anybody taken advantage of this yet? What are you using it for?

3 Likes
3

I’ve not done anything yet… but I’m excited to send some of my longer novels (100k+ words) and see if I can get some good info on them. Now I might be able to send a six book series… Hrm… I’ll report back once I test some!

And for anyone wanting to try, free tokens have been extended… see the thread here to read more and see if you’re eligible…

2 Likes
4

This is a developer (API) forum, but many enthusiast pro users could use a million-token window. It would be transformational for those attending the administrative court with hundreds of exhibits. :folded_hands:

1 Like
5

Didn’t do much with it yet as my specifics require laser focus on task and huge context chunks blur the attention on the task. But will definitely try to see if it can be used in doc generations based on large API definitions and database schemas. Currently, kind of sceptic if that will work in such a specific application, maybe will be a great surprise.

1 Like
6

Hey Edwin, just was playing with this documentation and from what I saw, unfortunately the large context window even if used only to provide context info, does not bring better results when used in specific tasks (several files concatenated, containing tech info) : task was to generate a documentation for an entity in the database- fields, purpose, translations, along with its relations and business logic.

Even if all the info was available in the context and the expected output was under 5k tokens, the GPT4.1 fails to consider the required context and zaps a lot of otherwise simple elements from it. Also the expected format was simple markdown, no schemas or anything complicated…

Update (almost an hour later), finally got it working by using o4-mini-high + files uploads + 30 min back and forth to explain certain points one by one to produce a documentation md file of 175 (regular) lines… The quality (again, mind that I’m aiming the interstellar precision, so don’t think GPT is so-so, way better than many employees I’ve seen in my career):

  • good working draft, still need some adjustments, but I’ve hit the limit in chat on 4.5 yesterday, so cant push it further with AI, only manually.

Usually, for this type of tasks :

  1. I go gradually from ground up, initially using the 3o to do dirty drafts based on high volume of data (those rarely fail reading files),
  2. then get more context injected piece by piece with 4.1,
  3. then restate on the outputs to get all data again (same 4.1 or 4o)
  4. then reinject the example output with notes about the formatting (hate current default GPT formatting), so that it gives me “final” version.
  5. And the little touch of magic (“nuances”) by 4.5 to finish.

But for large context, even that approach in chat is shaky, models loose focus on task input elements required and start summarizing the important input, event reasoning does not help on that level. BTW I think I have not tested the deep search in this particular flow in chat. Might actually produce better results if “locked” on provided files.

So, for now, I’m still not that convinced, I do see way better results with workflow breakdown and single simple tasks with smaller context via api + final “assembly” again via API.

Will post again if come back to this thing.

1 Like