If I want to add a file (e.g. PDF) along with my prompt as a message (NOT upload it for an assistant or fine-tuning), is there a way to do that through the API?
From what I’m seeing so far, one way to do it is send it all as text in the prompt, but the length limitations are far lower than those in the UI. A file that the UI handles without a problem, runs into token length issues through the API, even with the model with highest token count limit in the context window.
1 Like
Hi!
What do you want the models to do with that file? 
If it can’t read all that text, and isn’t supposed to index it, do you want the the model to work the code interpreter to interact with that file?
1 Like
I would want it to read the file (let’s say that contains pages of fiction text) and analyze or answer a question about it, just within that current interaction.
I don’t want it to be stored for knowledge retrieval, as my next interaction would include a new file with new content for answering a set of questions just about the new document.
1 Like
Hi and welcome to the Developer Forum!
At a fundamental level, the language model simply takes a series of input tokens and processes them into a number of output tokens. The only thing the model understands is those tokens, you must pass the model the text/tokens you wish to be processed.
Text goes in, text comes out. Anything else is a layer of abstraction, there are no files to be handled, you have to open the file read the text and insert that text into the prompt the model will receive, what ChatGPT does is hide all of that away behind the scenes in the form of you uploading a file, with the API you have to handle that yourself.
The Assistants API does do some of this for you if you wish to take a look into that here : https://platform.openai.com/docs/assistants/overview
2 Likes
Got it, I figured as much, as far as handling the file on my side and just passing it as a string within the content, along with the prompt. Where I hit the roadblock was with the token limit. I can run that same file in the UI without a problem. Through the API, I hit the token limit. It seems like that is, at this point, by design?
My alternative does seem to be going the assistants route so that I can actually upload a larger file. This is my understanding:
- Upload the file with purpose “assistants”
- Create the assistant. Do NOT include the file in the assistant object (As we don’t want the file for retrieval. The file will be different for each thread)
- Create a thread, with a message. Include the file id in the message object.
- Run the thread with the assistant.
Since the file is not part of the assistant, it will only be relevant for this thread, and not for retrieval. Am I correct in this assumption?
Thank you!
@arete , wondering if you had success with this path?
[EDIT: found the issue]
I figured the issue - when I setup the assistant I did not add file_search to the tools for the assistant, as I did not want to attache the file for the assistant in general, just on the thread. However, that appears to cause the assistant to then ignore the file attached on the message. Amending the assistant setup to
await openai.beta.assistants.create({name, instructions, tools: [{ type: 'file_search' }] }) fixes the behaviour I was seeing and seems to closely match the results I see when using the chatGPT UI directly.
[original]
Has anyone successfully achieved this?
I’ve tried with the steps outlined above (specifics below) but (with limited testing) am seeing very unexpected results - essentially the assistant is ignoring the file provided. So wanted to check before I investigate further.
(pseudo) code
- create a file:
openai.files.create({fileStream, purpose: "assistants"}) - create an assistant (without the file)
openai.beta.assistants.create({name, instructions })← see edit, needs file_search tool - create thread:
openai.beta.threads.create(); - create message:
openai.beta.threads.messages.create(thread.id, { role: "user", content: userMessage, attachments: [{ file_id: fileId, tools: [{ type: "file_search" }] }], }); - run and get response
anyone else managed to get this working / anything obviously wrong in above? ← yes, me 
One thought here if anyone on the OpenAI side sees this - given the apparent behaviour above, would be good to at least throw an error / warning if you do attach a file using a tool type which the assistant is not registered to be able to understand. (or better, from a devX perspective, to take the presence of a file with file_search on a message to mean that the assistant should use it, without requiring file_search to be registered during setup - but understand that could easily be a non-trivial change under the hood)
Breaking instead of ignoring would break a lot of threads and break a lot of user’s habits.
- threads are not connected to any particular assistant or its enabled features
- user messages of the past and current are not easily distinguished - they all may have attachments that can be 7-day-temporary vector store of a thread with further inquiry possible
- disabling file search would then be impossible, or switching to assistant without one, or using a thread past vector store expiration. There is no mechanism to remove the file id, vector store attachment to a thread etc.
Not 100% sure I follow that logic (which is likely my lack of depth of knowledge)
But I was talking here about a call to message.create, not thread where:
- you specify an attachment
- you specify a tool type
- you have an assistant id
- …and… that assistant does not have this tool.type setup
in that case, would anyone actually want the attachment they specified to be ignored?
Given (AFAIK) you cannot change a messages assistantId isnt a message ultimately always tied to a given assistant (even if a thread is not)?
Ultimately, totally get that this may be intentional design vs oversight. In which case 
- you create a message
- you specify an attachment
- you specify a file_search tool
- you ask a question
- you ask another question in 10 days -
500 server error: cannot process request - no vector store, because the attachment auto-created vector store is expired. - you switch to an assistant without file_search tool to try a workaround -
500 server error: user attachment requires file_search - etc…
Instead, assistants just passively forgets about user uploaded files as vector stores without error, and doesn’t mandate that past or present attachments “work” if you want to switch a chat to another assistant without.
When you find so many things counter to logic, reason, and budget, you may abandon assistants and write your own chat completions version.
It seems that attaching a PDF file in the API is now fully supported and there is no need for a workaround or using the assistants. The code in the link seems to work either through chat completion or the responses API.
2 Likes
watch out for token usage though …
1 Like