Converting PDF to Markdown with OCR

[jochenschultz]

Ask GPT-4o about a file - Example python function with file upload base64 and tiktoken and usage history with forced json return

Thought I post an example since it took some time to figure it out: settings = { 'openai_api_key': 'sk-.............' # Add your OpenAI API key here } # i am giving it a connection to my database and an upload id (when i upload the file I rename it to a uuid to have a unique id to use that as a key) def send_to_gpt4o(image_path, upload_id, conn): # Function to encode the image def encode_image(image_path): with open(image_path, "rb") as image_file: return bas…

I mean GPT4o is multi modal. It can take images as well as pdf files I guess.

So you base 64 encode the file and send it.

Of couse you should also change the prompt to something you are doing in ChatGPT normally.

But if you want to save on API cost I would suggest to use something like ghostscript to split the PDF in single tiff files and pytesseract to convert the PDF to hocr (in a loop over each tiff).

And then use GPT-3.5 with a prompt like

give me markdown from this hocr:

[hocr]