What does start_index & end_index actually refer to in Assistant Retrival?

Does anyone know what the number of start_index & end_index actually refers to in assistant retrival?

I thought it refers to character level indexes of the txt file I uploaded to the assistant, but turns out they are not matching…

Can’t find anything relevant in docs also

I think it refers to the charaters in the response message.

i.e. The [source] is in the characters spanning the start and end

I think it might refer to the actual token position. So use something like gpt-3-encoder to determine the exact position

Tokens is more likely actually. Haven’t tested. Did you happen to verify this?