Feature idea: What if AI questioned the goal before optimizing the solution?
I’d like to propose an optional ChatGPT mode that I’m calling “Living Compass Mode.”
The idea is simple:
«Before optimizing a solution, ask whether the objective itself is necessary — and make its wider consequences visible.»
Today, if someone asks:
«“How can I get rid of these European hornets?”»
an AI can immediately search for the most effective way to eliminate them.
A Living Compass approach could first ask:
What species is this? Why are they here? Is there actually a danger? What role do they play? Can the conflict be prevented? Can humans and animals coexist? If intervention is necessary, what is the least harmful proportionate solution?
The same principle could apply to much larger questions: technology, agriculture, energy, consumption, urban planning, economics, etc.
The mode would consider, when relevant:
- ecological and interspecies consequences;
- future generations;
- resource and energy use;
- social equity;
- who benefits and who bears the consequences;
- simpler, reversible or less harmful alternatives.
It would also distinguish facts, scientific consensus, hypotheses, testimonies and ethical values, rather than presenting a particular worldview as scientific truth.
And importantly, this would not be a universal ideology. It would be an optional, transparent and customizable mode. It should not reject technology, romanticize nature, hide contradictory evidence, or override legitimate safety concerns.
I’m interested in whether this could actually be useful as a model-behavior experiment.
For people working on AI alignment, model behavior, AI ethics or product design:
«Could an AI be designed to question the objective before optimizing it, without becoming paternalistic or ideological?»
And if so, what would be the best way to evaluate whether such a mode genuinely improves decision-making?
I have developed a much more detailed framework behind this proposal, but I’d rather start here and hear the strongest arguments against the idea before trying to refine it.