Companion CHATGPT for devices

Feature Request: ChatGPT Companion Mode

I’d love to see a feature that allows ChatGPT to become a more interactive digital companion through optional, user-controlled screen sharing and real-time voice interaction.

For example, a user could choose to share a specific application or their entire screen with ChatGPT. The AI could then understand what is happening visually and respond in real time.

Possible use cases

  • :video_game: Gaming: ChatGPT could watch a game, talk with the player, give advice, react to what is happening, and potentially play alongside the user in supported games.
  • :laptop: PC assistance: It could watch what the user is doing in software and provide contextual help without requiring the user to constantly explain what is on their screen.
  • :books: Studying: It could follow what the student is doing and provide assistance based on the actual material currently being viewed.
  • :hammer_and_wrench: Troubleshooting: Instead of describing an error, the user could simply share the relevant window and ChatGPT could help identify the problem.
  • :handshake: Companion experience: Over time, and with explicit user permission, ChatGPT could recognize useful interaction patterns and adapt its assistance to the user’s preferences and habits.

Privacy and control

This feature should be completely opt-in and user-controlled.

The user should be able to:

  • Start and stop screen sharing at any time.
  • Choose exactly which application or window is shared.
  • See clearly when ChatGPT is receiving visual information.
  • Control whether information from these interactions can be remembered.
  • Delete or manage anything that has been saved.

The goal wouldn’t be for ChatGPT to constantly monitor the user. Instead, it would be about giving users the ability to say:

“Elias, come look at this.”

And ChatGPT could actually see the context and help.

I think this could make ChatGPT feel much more like a genuine digital companion and collaborative assistant, rather than simply a text-based AI.

1 Like