Forbidden letters keep turning up (generating text for students)

Prompting
1

Hi, I’m trying to generate small texts for 1st grade students that only know a handful of letters. However the answers keep including forbidden letters. (At first I thought it’s a translation issue where it would generate the text in English and then translate to German without another check for the letters. But the same keeps happening in English, too.) Here’s my prompt and the answer:

You are an English teacher. You teach first grade reading in English. Please write a short reading exercise for the pupils of about 50 words. The words may only have the letters a, e, f, g, i, l, m, n, o, p, r, s, t, u. All other letters are disallowed. The words must not contain diphtongs like ei, au, eu. The words must not contain umlauts (ä, ö, ü, ß) or sch, ng, st, sp. Try to write a short scene or story that’s appropriate for kids, like playing in the park or about animals.

ChatGPT 4: In a park, a frog met a snail. The frog was green, and the snail was small. They were pals. The frog and snail liked to play games. Frog jumped, and snail moved slow. They had fun times in the park.

me: Check your text for forbidden letters and report all words containing forbidden letters, please.

ChatGPT 4: I apologize for the oversight in my previous response. Here are the words containing forbidden letters: met, small …
[notice how “met” and “small” are entirely allowed while “they” should be on the list]

Does anyone know if this is possible at all, and how to prompt it?

Thanks!

2

Chat is pretty good about critiquing its own work. I’ve had great luck asking it critique and suggest revisions to my prompt to make it clearer, especially if I do that after pointing out an error as you did above. just stay in a loop of gpt-produce, you correct, you ask from prompt revision, iterate, untill you are satisfied.

3

Oh - also, be specific. For example, point out that ‘they’ contains a y, which is not in the list of allowed letters. That way gpt knows which part of the prompt to focus on revising.

4

That’s not my experience here. I have done many many rounds of: GPT, please list all forbidden letters. […] Please check your generated text for forbidden letters. Then revise it that it only contains allowed letters. [“Sorry, I made a mistake” … proceeds to write out the exact same text again] Then I point out the words including forbidden words. [“Sorry”, and then the almost same text again]

If anyone cares to try, I guess my prompting is misleading somehow? But I’ve run out of ideas for now.

5

Yes, but have you asked GPT to critique or revise your prompt?

6

I’m not getting anywhere, unfortunately. Revise or suggest improvements lead to a different style (more like rephrasing for a human) but the same problem: “Try to write a short scene or story that’s appropriate for kids, like playing in the park or about animals.” becomes “Aim to create a short tale or scene fitting for young ones, such as playing in a park or about fauna.” No luck.