API for Playground for large file

I have used playground very successfully to get answers to questions. I have the code too. I want to loop over the questions(20,000) in an excel file and send them to the API? any suggestions welcome


import os
import openai
import openpyxl

openai.api_key = os.getenv(“OPENAI_API_KEY”)
wrkbk = openpyxl.load_workbook(“data_prep_combined”)
gc = wrkbk.active

for row in gc.iter_rows(min_row=1,min_col =1,max_row=20000,max_col =1):
For cell in row:

response = openai.Completion.create(
engine=“text-davinci-002”,
prompt=(cell.value(i,1))
temperature=0.7,
max_tokens=256,
top_p=1,
frequency_penalty=0,
presence_penalty=0
)
Would this work? is there a limit on number of rows( I have 20,000). I seem to run into issues.
Thanks

If this is the entire code, you are essentially overwriting the response with each query.
This will just have the completion for the very last prompt and just exit.
Wrap this in a function, append the responses to a list and return the list.

Something like the below code:

def xyz():
answers = []
for row in gc.iter_rows(min_row=1,min_col =1,max_row=20000,max_col =1):
    for cell in row:

        response = openai.Completion.create(
                        engine=“text-davinci-002”,
                         prompt=(cell.value(i,1))
                        temperature=0.7,
                        max_tokens=256,
                        top_p=1,
                        frequency_penalty=0,
                        presence_penalty=0
                )

        answers.append(response.choices[0].text)

return answers
2 Likes

Just to answer your 2nd question, No, there is no limit except financial! You might run out of tokens.

Thanks so much. Really appreciate it.

I’ve tried working with this code but continue to get the error that ‘i’ isn’t defined. The only way I can get a successful result is by changing the line from prompt=(cell.value(i,1)), to prompt=(cell),, but doing that results in only one completion.

What am I missing? Thanks for providing the code and any additional help you can offer.

I would be happy to help if you could share the excel file, or something formatted in a similar way (if your data is confidential).

Suggestion:
Try printing the cell value rather than using the completion call, just to debug if you reach each cell of interest.

Thanks for your offer to help, Vaibhav. I’ve since figured out what I was doing wrong and have moved on to other challenges :slight_smile: Will start another thread to discuss.