I hope this is ok to post.
I have a WordPress site and use a plugin called AI Engine to manage my OpenAI application. This links with Pinecone to store word embeddings - everything so far is ok! My problem is that I would like to embed quite large documents for my chatbot to reference so need to chunk them. I have split one article into 9 chunks of a few hundred words each with the titles being:
Article Title - Foreword
Article Title - Recommendation 1
Article Title - Recommendation 2
Article Title - Recommendation 3
down to recommendation 8. The problem is that my chatbot doesn’t recognise that these are linked so when a user asks a question about this article I get quite a disjointed reply.
What would be the best way to create embeddings for this? Thank you for your help, spent days on this and getting nowhere!