This is an interesting question. Let’s make documentation.
[Total tokens for 3-small: 2183 embeddings] Counted: 128026; sending now…
- Embeddings failure Error code: 400 - {'error': {'message': "'$.input' is invalid. Please check the API reference: https://platform.openai.com/docs/api-reference.", 'type': 'invalid_request_error', 'param': None, 'code': None}}`
So we know what fails - and what we don’t have to pay for. After finding success, simple thing to knock of an embedding from the list at a time and see the exact value…
2042:“[2039] It was just dark now. I never went near the”
match score: -0.1131
2043:“[2040] “Good lan’! is dat you, honey? Doan’ make n”
match score: -0.0350
2044:“[2041] It was Jim’s voice—nothing ever sounded so "
match score: -0.0612
2045:”[2042] “Laws bless you, chile, I ’uz right down sh"
match score: -0.0043
2046:“[2043] I says:”
match score: 0.0742
2047:“[2044] “All right—that’s mighty good; they won’t f”
match score: -0.0253
maximum embeddings batch list length: 2048
I’ll leave it up to you to max the thing out on 8k list items and see…