As the title stated. Instruction Strings are token. How about the document? What if I have a huge database in the uploaded document?
If upload document doesn’t count as Tokens, can I put part of the instruction in the document? like putting the formula in the file, and in the instructions I will just say run the “ABC formula” in the file?
No they do not count as token for sure. Because the text chunking, embedding and vector store is being handled by openai - at best embedding API cost maybe incurred. But tokens are not consumed by uploaded files. This is to the best of my knowledge.
This is an interesting question because if the result of the automated embedding and similarity search is above the similarity threshold (presumably the method OpenAI ‘decides’ to use it), then in order for that chunk of text to be ‘used’, it must be part of the prompt. So do those automatically added tokens count against the total?
@jaja1035 I don’t think this approach will work as the instructions in the document may not be seen by AI, because (presumably) the way it decides to use something is with the vector similarity search score, between the prompt and each chunk. If your prompt says something about “ABC formula” and a chunk has “ABC formula” then that chunk may get selected to be included in the prompt. But it may also NOT match and therefor not be used (maybe because all the stuff surrounding “ABC formula” throws the similarity search below the threshold (a setting that we do not have control over).