Force api response to be in non English language. How?

API
21

First results of incorporating context in the system role are pretty amazing!! I have also tried the hybrid approach but so far i cannot say which is better. The basic characteristics should definitely be described in the system role, i would say.

2 Likes
22

Glad to hear that.
As I always say: ā€œMake them speak EllinikĆ”. Donā€™t let them speak English.ā€ :grin:

2 Likes
23

You are the man! Ī ĪµĻ‚ Ļ„Ī± Ī¼ĪµĪ³Ī¬Ī»Īµ!

Btw, do you think I should mess with temperature and max_tokens or keep the default (donā€™t know what they are) values?

2 Likes
24

Donā€™t say ā€œmessā€ - say ā€œperform acceptance tests on a new technology.ā€

Max_tokens is easy, it controls the number of tokens (2/3 of an English word, approximately, but with too much variation) in the modelā€™s response. OpenAI charges subscribers per token. If you have a ā€œfreeā€ ($18 credit) account, the response tokens will consume a few cents of your credit. Since you made a chatbot using your API key, get used to checking your platform:
https://platform.openai.com/account/usage - OpenAI API

Temperature is the setting most discussed in this forum. I wrote a lot about it here. I like to call it a creative freedom degree and it has a range from 0 to 1 (officially or 0% to 100%, but it goes to 2 unofficially).
Since they are Language Models, they change the way theyā€™re talking by the probability of words in the same context.
A Temperature = 0 is very boring, they answer in the very same way, with the very same words.
The ChatGPT has Temperature = 0.7 fixed, so it has more freedom to look for less frequent words in the same context - I am against such a fixed temperature, and in my opinion, the temperature should change according to the (meta) context or subject. for example:

  1. If a user requests a poem for a rock song, the Temperature could be 0.8;
  2. If someone requests a solution for a Newtonian mechanics problem, it could be 0.5;
  3. If a mother requests healthcare advice for her child, it should be 0.3 or less;

OpenAI chose 0.7 for ChatGPT to make it creatively funny and popular, to dissipate fears about AI.

However, less frequent words in the same context is what the AI Engineers expected - and the neural network is not predictable, and the training dataset has a lot of influence on the responses. As I said earlier nothing in AI is as we expect.

I suggest you test your chatbot with different temperatures and accordingly your prompts. There is no temperature setting for all cases.
If you expect a serious and logical answer for your prompt keep the temperature low - but zero, I am sure you will not like it.
If your prompt requests a more creative answer, the recommended set is 0.5 or higher;
If you want to see a crazy AI set the temperature to 0.85 or higher.

As the temperature raises, the more creative answers, but less reliability on the modelā€™s answer - less logical or scientific - the model will open the range widely looking for more alternatives to provide an answer, that is more unpredictable.

That is the funny part of all of this. Welcome to AI.

2 Likes
25

One thing i noticed and i am not sure it is a coincidence is that when i gave all context via the user role, the api response is MUCH slower than when i went hybrid. This occurs only on open ended questions where the response is long. When i ask how much is 2 + 2 the response is almost instant. I need to test some more, but anyone can provide any insight regarding this?

Thanks a lot

Edit: I need to test more. Maybe the delay is relayed to the fact that I have set no max_tokens. It seems that both hybrid and ā€œall in system roleā€ approaches are facing LONG delays when the response is long.

Edit 2: I am totally confused. I really have no idea why i am facing these delays.

26

So it seems that the delays are smth other people are experiencing. Which is good because i probably am not doing anything wrong and bad because it seems that OpenAI needs to fix this resource issue.

27

There are many complaints about GPT-3.5 even the Turbo one - please check the forum.
Mainly because they are ā€œfreeā€ to use: they are the ā€œfastest freeā€ available, ChatGPT is a 3.5 now, some 3.5 models are used as a demo for 4 models, and plus the misuse of 3.5 models by inexperienced users.

Consider changing the model, sometimes. Anyway, I have a suggestion for you about your ā€˜hybridā€™ approach. It is a test:

  • Instead of filling System and User with the very same things, reinforce the power of the User prompt and pass its authority to the System, like this:
    User: 1. Follow the instructions and rules provided in the System role;
  • I believe this will work for most or all models including the ones with weak System power like the gpt-3.5-turbo-0301 you found in OpenAI documentation;

Please let me know the results.

1 Like
28

Thanks again for your valuable help.

ATM I have everything related to context in system role and it behaves pretty good.

EDIT: I am gonna try this approach as well, of course.

29

So i tried this and my first impression is that it seems to be working MUCH more consistently. Will get back with more. Thanks again for your valuable help @AlexDeM

1 Like