Chatgpt api maintain text throught a connection

API
1

I want to create some sql statements with chatgpt. for that I’m passing the table definition in each chat completion. So if I ask first generate me the sql query to get all orders, as part of my message I pass the table definition. Then if I want chatgpt to generate me a second query (e.g get all orders that were paid) I m passing the table definition again.
Is there any way to just pass only once the table definition and ask many questions until I end a session?

2

The API is a one shot.

So you have to maintain an array of the messages you want to send each time.

In your local code you need to maintain a history of past messages and append the new query at the end.

1 Like
3

Thanks for your reply but this is not what I asked for.

Question was about on how to preserve the table definition on chatgpt session without sending it in every message or this is not possible?

4

My response was entirely appropriate wrt the API.

Are you in the right Category? This Category concerns the API.

Are you instead referring to a session on the ChatGPT website?

If you are referring to the API can you please detail how you are “communicating the table definition” presently, using the API?

What does your code look like? And what does your message that includes the table definition look like?

1 Like
5

Yes I was talking about the api.
Each call is an http call. Is there any way preserving data (table data in my case) between http calls?

1 Like
6

No, hence the need to concatenate or maintain a “window” of messages each time.

You can prioritise inclusion into that array by algorithmically keeping your table definition message every time and then including a window of to and fro prompts and responses afterwards.

If that’s not important, then just memorise the table definition message and send it as the second entry in your array the next time again (the first entry being the system message), the last being your new query.

so the call might look like:

import openai

openai.ChatCompletion.create(
  model="gpt-3.5-turbo",
  messages=[
        {"role": "system", "content": "You are a helpful assistant expert in table definitions"},
        {"role": "user", "content": "My table definition is: CREATE TABLE ..."},
        {"role": "user", "content": "Is this correct SQL syntax?"}
    ]
)

But more likely you’d create that array ahead of the call in your code.

1 Like
7

Thanks for your reply.

Yes that is what I m currently doing but this is very inefficient and expensive.

I didn’t check how it sends request when using the browser version but I suppose is sending them again and again like the api needa to do right?

8

The only alternative would be to have gigabytes of GPU memory dedicated to just you and your current inference session, then though some creative use of model state snapshots, it may be possible, but you would be paying for an entire H100 compute node just for you, it would start to make economic sense at 450million tokens per day.

2 Likes
9

My suggestion here is just to use GPT 3.5 and avoid GPT 4 if you can as the latter is an order of magnitude more expensive.

Yep, you could look into a local solution with your own open source LLM, but then you have to factor in all your infrastructure costs …

10

Since Im new in the llm world, could you suggest some open source LLMs that can offer this functionality?

1 Like
11

How about?:

1 Like
12

If anyone is in doubt I can also confirm that Llama 2 is the Open Source choice everyone is using, and HuggingFace is where everyone gets the models from, so @merefield is correct.

1 Like