My main concern with the ChatML is the sheer number of “extra” tokens that will be consumed by all this “formatting”
Say what now?The cost is also $0.002/1000 tokens now… a 90% reduction… 
Granted
We are aware of this, nothing to share yet but hopefully this will improve.
it works if you use openai.ChatCompletion.create() call
old call was openai.Completion.create()
so this is for chat orientated usage
It seems like it’s not calculated as a whole string.
{"role": "system", "content": f"It is 2023-03-01 16:53:21. When the user asks a question that you don't know, condense & format the question into a query, prefixed with: QUERY|"},
{"role": "user", "content": "Hi"},
{"role": "assistant", "content": "Hi! How can I help?"},
{"role": "user", "content": "Hi. When was ChatGPT API released?"},
{"role": "assistant", "content": "QUERY|When was ChatGPT API released?"},
{"role": "system", "content": "ChatGPT API was released March 1, 2023. You may answer the question with this answer"}
This request is 201 tokens (If you copy and paste it all) using the openai page but was only counted as 125 prompt tokens in the api response
I’m getting this: AttributeError: module ‘openai’ has no attribute ‘ChatCompletion’
I was getting the same error but after upgrading my openai version, the error went away
We also learned key technical details with this release (which I have been testing like everyone else and loving it), summarized here:
chat method in the OpenAI API. Nothing wrong with this, just being clear. The API is still the OpenAI API but now with a new (very cool) chat method / endpoint.conversation array, if you like) each message tagged by one of three roles (system, user, assistant).chat method is not fine-tunable.chat method is currently fixed to a single model.This will come up soon and users will have problems and so here is a key implementation note:
If you try to use a different model than gpt-3.5-turbo you will get an error:
irb(main):080:1* response = client.chat(
irb(main):081:2* parameters: {
irb(main):082:2* model: "text-davinci-003",
irb(main):083:2* messages: [{ role: "user", content: "Hello!"}],
irb(main):084:0> })
=>
irb(main):085:0> response
=>
{"error"=>
{"message"=>"Invalid URL (POST /v1/chat/completions)",
"type"=>"invalid_request_error",
"param"=>nil,
"code"=>nil}}
Hope this helps.
I’m wondering the same thing. I couldn’t find any examples
This new endpoint is amazing. Uploaded some example code in Node for anyone who’s having trouble integrating.
Old generation of “an angry dwarf” (text-davinci-003):
Grievous was an angry dwarf. He had a scowl on his face and a piercing gaze that made others avoid his presence. His unkempt hair was full of tangles and his clothing was in tatters. His drooping moustache seemed to mimic his perpetual frown and he spoke with a growl, conveying all his bitter resentment through every word.
When someone did the slightest thing to annoy him, he would start yelling and stomping his feet. He kept a sharp eye on everything around him, looking for any sign of disrespect or trouble. He felt as if the world had done him wrong, so he kept all those around him at a distance, no matter how kind they were. His ranting about all the injustice in life served as an expression of his anger and frustration.
And yet, despite all this aggression, there was something sad about Gregg’s expression - as though he was desperately trying to make sense of the world around him, but all he found was pain and suffering instead of joy and peace.
New generation of “an angry dwarf”:
The dwarf stands at barely four feet tall, yet his fiery temper makes him seem larger than life. His broad shoulders are hunched forward in a stance that screams aggression, and his thick, muscular arms chisel with veins as if coiled to spring into action at any moment. His bushy eyebrows nearly meet in the middle of his forehead, furrowing deep with rage as he glares through piercing blue eyes. A thick beard and mustache frame his scowling lips, which are twisted in a permanent sneer as if daring anyone to cross him.
His clothing is utilitarian, consisting of roughspun trousers and shirt with heavy leather boots laced up to his knees. A sturdy leather vest provides some protection to his torso but doesn’t hinder his movements in the slightest. He carries an axe in one hand and a short sword in the other, both weapons gleaming with a deadly sharpness that reflects his intense focus on combat.
There’s a thunderous quality to the dwarf’s voice that booms like a clap of thunder when he speaks, emphasizing each word with a forceful punch. Even when he’s not yelling, there’s an underlying edge of anger that vibrates through every syllable. It’s clear that this dwarf has a short fuse and is not afraid to let loose with his formidable strength if provoked.
Despite all this, though, there’s a sense of honor and loyalty that lies beneath the dwarf’s anger. Once someone earns his trust, he’ll fight ferociously to protect them, even giving his own life if necessary. But those who cross him or betray him will find themselves at the receiving end of his fury, facing a force that seems unstoppable until justice is served.
The old one was good, but the new completion gives more fine control over tone and such.
I plugged it into my new Scratchpad tool and the fiction it outputs is amazing. Still wish context window was a little bit bigger, though!
Hi bro, really need your advice and input for this
I really impressed with the new ChatGPT API for 2 reason :
but I was focus on project to repurpose ( fine-tuning ) AI for special use.
My question, seem the ChatGPT Api not allowed fine-tuning
Thanks in advance, and thanks for your helpful presence in this community
Hi Sir,
I have been learning fine-tuning in GPT-3 (davinci) to create Chatbot Apps
( which train with product/company/ or even AMA(ask me anything) data
for many reason, 10% cost of Davinci made GPT3.5 a much better option for me.
how do I continue my project with GPT3.5(chatgpt API)? as it doesn’t allowed fine-tuning at the moment.
( but I see the example from OpenAI page, it seem it accept custom data )
If you want to get Whisper transcriptions in python and not deal with the overhead of the OpenAI python bindings, here is the python code I came up with:
import requests
import os
ApiKey = os.environ.get("ApiKey")
headers = {
'Authorization': f'Bearer {ApiKey}'
}
files = {
'file': open('/local/path/to/your/file/audio.wav', 'rb'),
'model': (None, 'whisper-1'),
}
response = requests.post('https://api.openai.com/v1/audio/transcriptions', headers=headers, files=files)
print(response.json())
You are welcome, @zhihong0321
Well, as for me, I think “tuning” has a distinct meaning in generative AI; so it’s not a term I would use, but I know what you mean.
Actually according to the chat method docs, it seems all the role keys contribute to the textual-information used by the API in a chat completion.
The API docs on this. point are awkaraedly worded, so it’s easy to understand how you might be confused. Yes, the current “MAX” for max_token for a `chat completion" is 4096 tokens.
You are welcome again.
Very great news ever! Thank you so much for this big news.
Just wondering if there’s ever going to be any plan to allow for custom roles? Finding it very restrictive for my particular use case