Optional Live Screen Awareness for ChatGPT

ChatGPT Feature requests
1

I would like to suggest a feature that would significantly improve ChatGPT’s usefulness as a personal assistant.

Feature: Optional Live Screen Awareness

Today, ChatGPT can analyze screenshots and uploaded images, but there is often a gap between what the user is seeing and what ChatGPT can verify. In many situations, the assistant can only analyze a static image and cannot determine whether elements appeared later, changed dynamically, or were missed by the user.

I would like to see an optional screen-awareness mode where the user can explicitly grant temporary permission for ChatGPT to view their screen in real time.

Potential benefits include:

• Troubleshooting apps, settings, and devices
• Assisting with software and website navigation
• Understanding dynamic interface changes
• Helping with image generation and creative workflows
• Travel booking and scheduling assistance
• Accessibility support
• Reducing the need for repeated screenshots

A recent example involved AI-generated images where additional image variations appeared in the interface. ChatGPT could not determine whether the images were present from the beginning or appeared later because it could only analyze screenshots I provided. Live screen awareness would have allowed the assistant to answer immediately and accurately.

Privacy and Control Requirements:

• Screen sharing must always be optional.
• Users must explicitly enable it.
• A clear visual indicator should show when screen awareness is active.
• Users should be able to stop sharing instantly.
• No screen access should occur without active user consent.

This feature would make ChatGPT feel less like a tool that analyzes snapshots and more like a true assistant that can understand what the user is experiencing in real time while still preserving privacy and user control.

Thank you for considering this suggestion.

2

Welcome to the community and thanks for sharing this, @DwightTurner.

The interesting part is that some of what you're describing actually exists today, at least on supported mobile devices. During Voice Mode, subscribers can choose to share their screen with ChatGPT, allowing it to see changes on the screen in real time rather than relying only on static screenshots. Screen sharing is optional, can be stopped at any time, and uses the device's built-in screen-sharing controls and indicators.

That said, your suggestion still highlights a real gap. Current screen sharing is tied to Voice Mode and supported platforms, and many users don't realize it's available. There are also situations where people are working from screenshots or desktop workflows and can't easily give ChatGPT live context.

Your privacy and control requirements also align closely with how the current feature is designed:

  • Explicit user activation
  • Visible screen-sharing indicators
  • Ability to stop instantly
  • No access unless sharing is actively enabled

So this feels less like a brand-new capability and more like an opportunity to expand and improve live screen awareness across additional platforms and workflows.

We'll also make sure this feedback is logged with the team. The examples you provided around troubleshooting, accessibility, creative workflows, and understanding dynamic UI changes help illustrate the value of broader screen-awareness support.

Appreciate you taking the time to write this up and provide a concrete use case.

-Mark G.