Using OpenAI API for Document Analysis with Static SOPs

API
1

I have a business process where I review three short documents together to answer specific questions. To streamline this, I’ve been using ChatGPT by providing it with the content of the documents and our SOPs (one SOP per question to be answered) as prompts. This setup works well, and I’m basically always getting the correct answers. Huge time savings. To streamline further, I’m currently developing a Python script to leverage the OpenAI API for this task, but I’ve never worked with the API before and I have a preliminary question . . .

Our SOPs is that they are mostly static but occasionally get updated with examples for rare situations or clarifications. These updates usually add a few bullet points per week. Given this, I have a couple of questions:

First, is it necessary to include the SOPs in every API request as background information? Considering the cost, it would be ideal to avoid providing the same information repeatedly.

Second, is there a way to load these SOP documents as a permanent reference for the API, so I don’t have to include them in every request?

I appreciate any insights or advice on how to optimize this process and reduce costs. Thank you in advance!

1 Like
2

Welcome to the dev community!

Yes, there’s no “state,” so you need to include whatever context you want in each call. (Shows how much value you can get out of ChatGPT!)

What you might do is look into RAG (retrieval augmented generation)… You can search here on the forum or the OpenAI Cookbook site. Basically, you vectorize chunks of text. When you’re putting together a prompt, you would only grab those chunks that are “relevant” for the rest of the query.

1 Like