Using gpt-4 API to Semantically Chunk Documents

API
52

So, this is the json file I end up with in order to do my chunking:

https://s3.us-west-2.amazonaws.com/docs.scbbs.com/docs/test/article11-out.json

Chunking so far works perfectly using the generated start and end line numbers.

However, I do have situations where the token count exceeds the limit (in this case 600) I have set.

This means I need to run ANOTHER prompt to semantically chunk it down even further (If I still don’t want to do the numeric chunking).

           {
                "title": "E. ARBITRATION OF DISPUTES CONCERNING CREDIT PROVISIONS",
                "level": 2,
                "token_count": **1355**,
                "start_line": "line0267",
                "has_children": "N",
                "children": [],
                "end_line": "line0326"
            },

https://s3.us-west-2.amazonaws.com/docs.scbbs.com/docs/test/13_Article_11_1.pdf#page=14

                    {
                        "title": "5. Selection of an Arbitrator; Place of Hearing",
                        "level": 3,
                        "token_count": **767**,
                        "start_line": "line0375",
                        "has_children": "N",
                        "children": [],
                        "end_line": "line0444"
                    },

https://s3.us-west-2.amazonaws.com/docs.scbbs.com/docs/test/13_Article_11_1.pdf#page=18

                    {
                        "title": "6. Timeline; Citation of Expedited Arbitration Awards",
                        "level": 3,
                        "token_count": **714**,
                        "start_line": "line0445",
                        "has_children": "N",
                        "children": [],
                        "end_line": "line0458"
                    },

https://s3.us-west-2.amazonaws.com/docs.scbbs.com/docs/test/13_Article_11_1.pdf#page=20

I am thinking, at this point, I could send the model just the chunk with the line numbers, and ask it to return a json of just the suggested semantic sub-chunks of the submitted chunk.

I don’t want to re-do what I’ve already done which works. I just want a way to address this particular issue which may/may not arise.

Any prompt suggestions? I have my own ideas, but interested in how others would approach this.