Product Proposal: Let an Existing ChatGPT Conversation See My Shared Screen
Summary
I would like ChatGPT to support continuous, user-controlled screen or window sharing inside an ordinary existing Chat conversation.
The feature I need is not ChatGPT controlling my computer.
It is my existing ChatGPT conversation being able to see the same working context I can see while we continue talking normally.
I may later hand an implementation task to Work, Codex, or another agent. But before there is a task to execute, I often use ChatGPT to understand what I am seeing, identify the actual problem, challenge assumptions, explore alternatives, and formulate the task correctly.
That reasoning stage currently has a significant missing input: Chat cannot see the environment we are discussing unless I manually transfer pieces of it.
The Current Problem
My workflow often begins and lives in ChatGPT.
I may be working in PowerPoint, a browser, an application I am developing, an IDE, a spreadsheet, or several of these at once. While working, I use an ongoing ChatGPT conversation as a thinking partner.
Today, if I want ChatGPT to understand what I am seeing, I have to become the interface between my desktop and ChatGPT.
I copy text. I describe what is happening. I take screenshots. I upload them. I switch applications. Something changes. I take another screenshot. I explain what happened between the two.
In effect:
The user becomes ChatGPT’s perception API.
This creates more than inconvenience.
It reduces the quality of the shared context.
A screenshot transfers an artifact.
Continuous screen context transfers the situation.
I may not know which detail on the screen is important. If I knew that already, part of the reasoning problem would have been solved. Something I don’t think to capture may be exactly what ChatGPT would have noticed and asked about.
The Workflow I Want
Imagine I am using an application I am developing.
I share that application window with my existing ChatGPT conversation.
I continue using the application normally while talking or typing to ChatGPT.
I say:
“Something about this interaction is irritating me. Look at what happens when I do this.”
ChatGPT can observe what I am observing.
We discuss it.
I try another path through the interface.
ChatGPT notices a behaviour or inconsistency I had not considered.
We determine that the apparent problem is actually caused by something else.
Only after that conversation do we arrive at a concrete change worth making.
Now we can formulate a precise implementation brief and hand it to Codex, Work, or another execution system.
After implementation, I can return to the same conversation, use the changed application, and discuss the result while ChatGPT can see it.
The workflow becomes:
OBSERVE → CONVERSE → DISCOVER → UNDERSTAND → SPECIFY → EXECUTE → OBSERVE
The important point is that execution occurs relatively late in this process.
Why Work or Computer Control Does Not Replace This
Agentic tools solve a different and valuable problem.
Once I know what needs to be done, I can ask an agent to perform work.
But many important tasks do not begin with a well-formed instruction.
They begin with:
“Something here isn’t right.”
Or:
“Look at this. Why is this happening?”
Or even:
“I don’t know whether there is a problem here yet.”
The conversation is how the requirement emerges.
Moving to an execution-oriented environment too early reverses that relationship. It assumes I have already formulated the task that I am using ChatGPT to help me discover.
I don’t want Chat to become Work. I want Chat to gain sight.
The Minimum Useful Version
This could initially be entirely read-only.
Allow me to explicitly share:
- one application window,
- one monitor, or
- another clearly bounded visual surface
with my current ChatGPT conversation.
While sharing, ChatGPT should be able to perceive changes to that surface as they occur while our normal text or voice conversation continues.
The existing conversation context should remain intact.
I should be able to stop sharing immediately.
ChatGPT does not need mouse or keyboard control for this feature to be transformative.
Perception and action should remain conceptually separate:
Seeing something does not imply permission to operate it.
If computer control is added or connected later, that can have its own explicit authorization model.
Why Existing Conversation Context Matters
Starting a separate visual or agentic session is not equivalent.
The value of an established ChatGPT conversation is the context already accumulated within it: the project, previous decisions, terminology, constraints, rejected approaches, design philosophy, and the reasoning that led to the current state.
I don’t want to repeatedly reconstruct that context merely because the next piece of relevant information happens to exist on my screen rather than inside the ChatGPT window.
The ideal experience is extremely simple:
I am talking to ChatGPT.
I choose “Share screen/window.”
The conversation can now see what I am referring to.
We continue the same conversation.
That continuity is the feature.
Why This Matters Beyond Software Development
The same interaction applies anywhere the user’s real work is visual or distributed across applications.
A person could discuss a presentation while editing it, examine a spreadsheet while reasoning about the numbers, review a design while adjusting it, troubleshoot software while reproducing a problem, compare documents, study material, navigate an unfamiliar application, or simply ask:
“What am I missing here?”
The value isn’t primarily automation.
It is shared situational context between the human and the conversational model.
Product Principle
ChatGPT already has powerful reasoning capabilities.
Agents increasingly provide powerful execution capabilities.
There is an important layer between them:
shared observation.
A conversational system that can participate in understanding a situation before being asked to act on it can help users formulate better questions, identify better problems, and ultimately give execution agents better instructions.
This is therefore not simply a convenience feature for avoiding screenshots.
It expands what a ChatGPT conversation can reason with.
The Request
Please allow an existing normal ChatGPT conversation to continuously perceive a user-selected screen, application, or window while the user continues that same conversation.
Keep the sharing explicit, visible, revocable, and permissioned.
Computer control can remain separate.
Execution can remain with Work, Codex, or other agents.
Give Chat eyes.
Let the human and ChatGPT understand what should be done together.
Then let the appropriate tools do it.