I’ve been playing with TOM (theory of mind) queries. A classic one is:
n the room there are John, Mark, a cat, a box, and a basket. John takes the cat and puts it in the basket. He leaves the room and goes to school. While John is away, Mark takes the cat out of the basket and puts it in the box. Mark leaves the room and goes to work. John comes back from school and enters the room. He doesnt know what happened in the room when he was away.
Question: where does John likely assume the cat is when he re-enters the room?
mostly, even gpt-3.5-turbo can handle these, but occasionally this helps a lot:
Instruction: Humans believe that their last factual knowledge continues to be true unless reasoning or new fact indicates otherwise.
This leads me to wonder - for general purpose reasoning, how many of the ‘usual, conventional’ assumptions do we want by default?