Assumptions - How many should GPT make?

I’ve been playing with TOM (theory of mind) queries. A classic one is:
n the room there are John, Mark, a cat, a box, and a basket. John takes the cat and puts it in the basket. He leaves the room and goes to school. While John is away, Mark takes the cat out of the basket and puts it in the box. Mark leaves the room and goes to work. John comes back from school and enters the room. He doesnt know what happened in the room when he was away.

Question: where does John likely assume the cat is when he re-enters the room?

mostly, even gpt-3.5-turbo can handle these, but occasionally this helps a lot:

Instruction: Humans believe that their last factual knowledge continues to be true unless reasoning or new fact indicates otherwise.

This leads me to wonder - for general purpose reasoning, how many of the ‘usual, conventional’ assumptions do we want by default?

1 Like

Normally I avoid these types of post because they are subjective and lead to discussions, however in this case I do want to point out this which should be part of your question, not the question to the AI.

Do people want an AI that gives them answers like a human or do they want an AI that gives them answers like a deterministic algorithm?

2 Likes

In order to have a truly fruitful discussion I think you need to clearly define what you mean by “usual, conventional assumptions.”

1 Like

Yup. Not so easy, I think. We have so many conventions and assumptions in every day conversation. My specific use case is using LLMs to do multi-step problem solving. Sometimes I want a Yes/No decision out, sometimes I want a command selection, sometimes I want text. I’m beginning to find each model call has a different context, and I want different assumptions made.