Control chunk size when adding files to a Vectorstore for the new Assistant?

I’m using the OpenAI API to create a Vectorstore for the new Assistant. While I can easily add files to the Vectorstore, I haven’t found any information in the API documentation on how to control the chunk size used when vectorizing the document during the file addition process.

I’m interested in exploring different chunking strategies. By controlling the chunk size, I hope to optimize the Vectorstore’s performance and retrieval accuracy.

If you have any guidance on how to manage chunk sizes when creating a Vectorstore using the OpenAI API, I would greatly appreciate your help

Hi!

It is currently not possible to change the chunking strategy.
From the docs:

We have a few known limitations we’re working on adding support for in the coming months:

Support for modifying chunking, embedding, and other retrieval configurations.

The current state is described here:
https://platform.openai.com/docs/assistants/tools/file-search/how-it-works

So, it’s on the roadmap!

1 Like