We can argue over what’s considered a hallucination or not and you’re correct that’s a reasonable response to give a person. GPT-4 would probably give something reasonably similar… If you’ve never seen your system hallucinate anything I applaud you. I see it daily 
As to whether telling it to say each step does anything, in my experiments it does. If you look at my post on Better Chain-of-Thought I have the model solving complex math problems using the same style of zero shot Better CoT prompt I proposed here. It helps when it can see itself think in detail…
And you’re correct, people want answers not to see the machine think. That’s why I instruct it to prefix a <response> marker just before it responds. You can just search for that marker to find the final answer. Everything before that marker is just the model thinking and can be ignored.