Follow-up question with lower tokens possible?

ChatGPT
1

Hello,

I have the following issue.
I send a book via api user role to chatgpt and programming a first answer via assistant role. That perfectly works. But now, I want to ask follow-up questions, dependent on the answer chatgpt gives me.

This is possible when sending all again, the complete book, the answer and the follow-up question. But this leads to a lot of tokens, since the whole book has to be sent again for every follow-up question.

Is there a possibility, that I do not need to send the book again and again for every follow-up question?

Thank you!

2

Welcome to the community.

If you want the content to be included/used, it’ll need to be in the prompt.

Is it possible to create a summary of the book then use that in subsequent runs?

3

No, it is a scientific book with formulas, etc. so makes no sense to make a summary…

4

Yeah, you’ll have to pay for it, then if you want it in the context.

Have you tried using a smaller model to cut prices?

5

Not yet. But I dont understand it right. If you use the payed version (no api), you can upload a book and ask questions right? So this is included in the $20 ? You could do it every day, which would be about $5 per day or whatever. Or is then happening something other in the background?

However, I hope this will be implemented soon.

6

Yeah, ChatGPT Plus you get a lot more value as you’re not charged per token. There are usage limits, but OpenAI is “eating” a lot of the costs themselves to make it available to many people (and gain market share…)

For the API, you pay for all tokens in and out, but you get more control.

7

Yes, but also the answer speed is completely different. Does a follow-up question for ChatGPT Plus to a book, really have the same time delay as for the first question? I’m not sure, but I would wonder if it is.

And if it is not, then there must be a difference between follow-up questions ChatGPT Plus vs API.

8

Hello, I have the following code and for the first question it works, but I do not get an answer for the follow-up question. There is no content in response2. Anyone knows why response works but response2 does not?

Thank you!

response = client.messages.create(
model = LLMchosen,
max_tokens = 2024,
temperature=0.0,

    messages = [
        {"role":"user","content":book},
        {"role":"assistant","content":askLLM[0]}
        ],   
)

print(response.content[0].text)

follow_up_question = input(“Do you want to ask anything else?”)

previous_conversation = book + askLLM[0] + response.content[0].text

if follow_up_question:
response2 = client.messages.create(
model = LLMchosen,
max_tokens = 2024,
temperature=0.0,
messages = [
{“role”:“user”,“content”:previous_conversation},
{“role”:“assistant”,“content”:follow_up_question}
],

)
print(response2.content[0].text)