Graded reader prompt not giving expected results

[mungbean0123]

Greetings,

I want to be able to create a graded reader for kids using only the words given in a list. I’m getting some results but not what I am looking for as ChatGPT is using tenses, plurals, and contractions not asked for. I’m new to this and I know I’m not getting the prompt correct. This is the prompt I am using…

write a 50 word story using only these words: “I”, “a”, “the”, “go”, “in”, “is”, “me”, “to”, “can”, “it”, “you”, “see”, “get”, “are”, “then”, “give”, “him”, “here”, “ran”, “has”, “she”, “with”, “boy”, “who”, “we”, “and”, “like”, “did”, “run”, “for”, “was”, “on”, “at”, “as”, “if”, of, not, have, us, had, her, make, old, ride, tell, up, will, “why”, “am”, “after”, “all”, “be”, “big”, “can’t”, “car”, “do”, “day”, “fun”, “give”, “his”, “new” , “your”, “my”, “went”, “how”, “look”, “now”, “out”, “saw”, “that”, “want”, “this”, “play”, “but”, “animal”, “about”, “best”, “come”, “down”, “friend”, “from”, “were”, “good”, “house”, “little”, “no”, “when”, “said”, “what”, “jump”, “made”, “off”, “over”, “some”, “there”, “they”, “school”, “thing”, “where”

Can anyone help me or point me to something I can read to fix this?

Thank you

[ruby_coder]

@mungbean0123

Can anyone help me or point me to something I can read to fix this?

Hi @mungbean0123

I was not able to engineer a prompt (in 10 mins of trying) which worked well for ChatGPT because we cannot set the temperature of a ChatGPT query (that I know of). When the temperature is too high, ChatGPT (the model) will get creative and more random, In your case too creative.

But if we lower the temperature, it is possible to get better results, but I only tested two completions:

However, using the OpenAI API, I slightly changed your prompt using the following params:

Revised Prompt

write a 50 word story using only words from the follow array of word: [“I”, “a”, “the”, “go”, “in”, “is”, “me”, “to”, “can”, “it”, “you”, “see”, “get”, “are”, “then”, “give”, “him”, “here”, “ran”, “has”, “she”, “with”, “boy”, “who”, “we”, “and”, “like”, “did”, “run”, “for”, “was”, “on”, “at”, “as”, “if”, of, not, have, us, had, her, make, old, ride, tell, up, will, “why”, “am”, “after”, “all”, “be”, “big”, “can’t”, “car”, “do”, “day”, “fun”, “give”, “his”, “new” , “your”, “my”, “went”, “how”, “look”, “now”, “out”, “saw”, “that”, “want”, “this”, “play”, “but”, “animal”, “about”, “best”, “come”, “down”, “friend”, “from”, “were”, “good”, “house”, “little”, “no”, “when”, “said”, “what”, “jump”, “made”, “off”, “over”, “some”, “there”, “they”, “school”, “thing”, “where”]

Params

Completion 1

Completion 2

Hope this helps.

[AlexDeM]

The ChatGPT cannot have its “temperature” controlled by a user or by itself through the chat interface - and it is set to 0.7 as default. It learns in real-time but it can’t hold information from previous requests from the same conversation except when the user keeps the thread subject continually for some turns in the conversation (short memory) - even so, some results seem to be “hallucinations” as the developers usually refer to some unexpected responses. The ChatGPT was pre-trained and its training was cut off on September 2021 - it is capable to improve interactions with humans.

On other hand, the OpenAI API engines, such as “davinci” and other models or codex (a framework of more than one model), can have their “temperature” controlled by the users. The OpenAI completion APIs models and engines can be trained on datasets the user provides, such as a word list.

Temperature is a hyperparameter that controls the randomness or creativity of the generated text with a range of 0.0 to 1.0 (official information). A temperature value between 0.0 and 1.0 (excluding 1.0) would result in a mixture of deterministic and random output, with lower values resulting in more conservative and predictable responses, and higher values resulting in more diverse and creative responses.

Temperature = 0.0 would result in the most deterministic output possible, where the model always selects the token with the highest probability at each step.

Temperature value = 0.1 to 0.3 will result in more predictable and conservative output. The model will be more likely to generate a high probability following words based on the preceding context.

Temperature = 0.7 is a commonly used default value, which balances between predictability and creativity - however, a model or framework referred to as different frameworks may have different default values. This value allows the model to generate text that is reasonably predictable but still contains some surprising and diverse elements. ChatGPT DEFAULT temperature value is 0.7 fixed constant.

Temperature = 0.9 to 1.0 (excluding 1.0) will result in more diverse and surprising output. The model will be more likely to generate unexpected and creative following words based on the preceding context.

Temperature = 1.0 would result in the most random output possible (??? - see below), where the model selects tokens with a distribution of probabilities according to their likelihood. Using a high-temperature value can sometimes lead to strange or unexpected outputs, as the model is freer to generate text that may not necessarily make sense or be coherent. It’s always a good idea to experiment with different temperature values and see what works best for a specific use case.

Temperature = 1.0 to 1.5 (unofficial information) is not within the expected range and may result in unpredictable or undesirable outputs - highly “hallucinatory” if it is not an error in some API engine or model.

It is suggested to use APIs to achieve a more deterministic goal using a lower temperature value such as 0.1 to 0.3 - such as creating a text limited to a word list. The API key and instructions for the use of OpenAI APIs models and codex are available on the official OpenAI GitHub page.

[ruby_coder]

@AlexDeM

It is suggested to use APIs to achieve a more deterministic goal using a lower temperature value such as 0.1 to 0.3

That is exactly what I just did above and not only in words, but in a full demonstration.

[AlexDeM]

Yes, I saw it. When I got to this thread I started to edit the text as it was mentioned:

@mungbean0123

point me to something I can read to fix this?

I got part of the text from the comments of a small VBA (for Excel) program I am developing for research. The editing and testing on ChatGPT took more than one hour, and when I came back I saw your reply - I didn’t know if the text would be useful but I didn’t want to waste my time - I just pasted the text, sorry if I couldn’t mention your reply. Anyway, you gave a solution - I like to think I gave something to read. I didn’t open the images you posted yet and my free account, my (lack of) full access, and my inexperience don’t allow me to engineer anything in 10 minutes around here… or anywhere after 1 AM, if that matters. I have a question: which interface are you using in your images? Playground? That is what I am trying to do in VBA but I can’t see how could I get such a level of detail including token counting, etc.

[sps]

Hi,

Here’s an experimental prompt I came up with that worked for chatGPT.

Being very intentional, concise and explicit with prompt should help.

But as others suggested, using other models with lower temperature will help achieve consistency and conformity.

[AlexDeM]

I made 12 of that, trying to training ChatGPT using different requests, rephrasing, ironic, polite, angry, concise… most of them started ok but at the end 2 or 3 unlisted words.

[ruby_coder]

@AlexDeM

have a question: which interface are you using in your images?

Hi @AlexDeM

No, not the Playground.

It is a app I wrote to access and experiment with all OpenAI API methods.

TBH, I think I used the Playground only once a few months ago.

[AlexDeM]

Something funny about this thread - I left for a few minutes to read the New list, I tried to come back, and the author deleted this topic. I got here with your reply link.
Your app is a very good job. I visited Playground 2 or 3 weeks ago - it was a bit disappointing. Did you write for your usage only? Or you make it available on GitHub?

[ruby_coder]

@AlexDeM

Did you write for your usage only?

Yes, so far. I’m waiting for the ChatGPT API to finish it.

I currently use it the app to help community developers here See more on that below.

The reason I wrote it was, at least for me, there was (and still is) too much noise, misinformation, “client fishing”, and solution “hand waving” going on here and not enough real “developer” facts and experimental tests where people actually using their own code and member data to test situations and credibility answer member questions.

Not that it matters, but I am heavily biased toward those in a developer forum who answer and “speak” though code and real-world tests and experiments and not just blah, blah, blah “hand save” opinions out of thin air, without solid testing.

See;

OpenAI Lab: How I help OpenAI Community Members Here

Hi Everyone! First, I want to thank those who have contacted me here and on LinkedIn with all the kind words about how I help and assist members and users here! It’s very gratifying to be held in such high regard by so many developers here. THANK YOU. LinkedIn is the best place to contact me, as I have disabled direct messages here temporarily (except to a hand full off long time members, staff and regulars). So, what I have been doing with my current OpenAI project? Lately, I have been c…

[AlexDeM]

Great job!!! It is a complete support system fitted for this technology. In my limited view I was supposing it was an user interface only. Amazing. Congratulations.

[ruby_coder]

@AlexDeM

Great job!!!

Thanks! Very kind of you to say so @AlexDeM

Yes, it covers just about the whole API enchilada and some add ons etc.

I like helping people by running their issues and testing v what is often seen here - a lot of what we called “hand waving” in engineering school decades ago.

Take care!

[mungbean0123]

Thank you so much for the help :0)

[mungbean0123]

Thank you everyone who contributed to this thread, I’ve learned so much!