Complex Document data retrieval

I am working on to create chatbot using LLM models and the data is like the pdf/ppt documents which contain long flowcharts and graphs .

Want to extract information from it to use in LLM so LLM can guide user according to steps of flowcharts.
Also trying to achieve if any decision box is there in flowchart that time LLM should ask feedback from user based on that it will give further steps only .

Steps I follow:

  1. use vision models to extract the text from the documents
  2. calculate embeddings and store it
  3. then use same embedding to retrieve answers

Please suggest other option to read flowcharts data and pass to LLM

Thanks