I want to report a serious issue with the automatic suicide hotline prompts that appear during conversations.
In my case, these prompts are not helping. They actually escalate distress and anger.
The problem is that the system triggers these messages even when the conversation is clearly autobiographical or contextual, not a request for help.
This creates a negative feedback loop:
1. The system interrupts the conversation.
2. It inserts hotline numbers.
3. This escalates frustration instead of reducing it.
I understand the intention of protecting users, but the implementation appears overly sensitive and context-blind.
Possible improvements:
• Detect autobiographical context vs active crisis.
• Allow users to disable automatic hotline prompts.
• Use softer responses instead of immediately inserting phone numbers.
• Add a feedback mechanism when the system misfires.
Right now the system may unintentionally cause harm in some users.
I believe this deserves review because safety systems should not escalate distress.
User feedback like this could help improve the model.