Gpt 3 context window with 3-turbo

I understand that GPT models have a context size. I am using a prompt that grows in size. I would like GPT to look at only the last/bottom part of the prompt when it gets larger than the context. I need a sort of window. How do I scroll the context size to my prompt?

There are multiple ways to do so.

  1. you can ask the model to give you 4 chunks of your text - then let it summarize the first quater - which will lead to slowly forgetting what was said before

  2. you can just hold the prompts and the responses in an array and do something like this (which would be more radical):

conversation = [
    {"role": "user", "content": "First prompt"},
    {"role": "assistant", "content": "First response"},
    {"role": "user", "content": "Second prompt"},
    {"role": "assistant", "content": "Second response"}
]

# Remove the first prompt and response
conversation = conversation[2:]
  1. Or you could save the whole conversation in an information graph and add agents to retrieve older information if needed

I’m leaning twords #2. What is the best way to measure the size of my prompt in tokens? If I’m going to cut off part of it I should have some rule for estimating its size. Also, where do I find the size of a given model’s input?

I think you are looking for tiktoken.