How to properly manage SSH access for MacOS Codex with Sandbox and Approve for me

How can I give the agent a permanent permission to use the ssh keys and connect to remote host while using “Approve for me”?

Most of the time I feel I need to explicitly tell in chat that agent has permission to connect outside the sandbox. Is there some easy way to avoid this?