Hi,
I have a book (as a single .tex file) containing a number of images. I would like to use the book with openais assistent API so that the vision model can answer questions about the images, but I don’t know how to “tell” openai which images go where in the book. I.e. that when the *tex source contains:
you can see more in the figure below:
\includegraphics[width=0.5\linewidth]{overview_figure}
it should refer to the image overview_figure.png(.pdf).
My understanding is that if I just compiled it to PDF format this occurs fairly seamlessly, however, I would prefer to upload my LaTeX source files since otherwise OpenAI tends to make mistakes in the equations.
A procedure I have though about is to simply use the API to create a description of each figure and then insert that in place of the figure along with labels/captions, however, I figure that I get better responses if the assistant has access to the vector corresponding to the figure.
Does anyone have tips/tricks for how to combine images with source text that refers to the images by name such as markdown or LaTeX?