Dear OpenAI Product and Research Teams,
I would like to propose a product concept that I believe could significantly change how people interact with AI agents and use them for real-world work.
Concept: A Multimodal AI Workspace
Instead of treating ChatGPT as a window that users occasionally consult, imagine turning it into a persistent workspace where the human and the AI agent work together on the same visual environment.
The core idea is simple:
The user communicates with the AI through voice, text, touch, pen and gestures, while the AI simultaneously understands and modifies the user’s active workspace.
The interaction could span a phone, one or multiple monitors, and the user’s existing applications.
Example setup
A user could connect their phone to a computer or monitor.
The phone could act as:
- A microphone for continuous voice interaction
- A text/chat interface
- A remote control for the AI
- A way to review or approve actions
The monitor(s) would provide the visual workspace.
For example:
Single monitor:
- 80–90%: the user’s actual work
- 10–20%: ChatGPT conversation / agent status
Two monitors:
- Monitor 1: primary application and work
- Monitor 2: ChatGPT, agent context, task history, previews and controls
The user should be able to dynamically change this layout.
The important part: spatial interaction
The user should not always have to describe an object verbally.
They could simply point at it.
For example, while working in an animation application:
The user says:
«“Make this camera movement about 30% slower.”»
The AI identifies the relevant animation and modifies it.
The user then draws a circle around a section of the timeline with a pen and says:
«“I’m talking about this part. Make the transition smoother.”»
The AI uses both inputs:
- What the user said
- What the user physically pointed at
It then modifies the correct element.
The same interaction could work with:
- CAD
- Architecture
- 3D modeling
- Animation
- Video editing
- Photoshop
- Excel
- PowerPoint
- Programming IDEs
- Web development
- Data analysis
- Research
- Writing
- Project management
Why this is different from traditional AI chat
Today, a user often has to translate what they see into language before the AI can understand it.
For example:
«“The third box from the left, below the blue button, needs to move slightly upward…”»
This is inefficient.
A multimodal workspace would allow the user to simply point at the box and say:
«“Move this up.”»
The screen itself becomes part of the language between the human and the AI.
This changes the interaction model from:
Human → Text/Voice → AI → Answer
to:
Human ↔ AI ↔ Shared Workspace
The AI should understand the state of the workspace
The agent should not only receive screenshots.
Ideally, it should understand:
- Which application is active
- Which windows are open
- Which objects are selected
- Cursor position
- Pen/touch coordinates
- Screen and monitor topology
- Relevant document/project context
- Current task
- Previous actions
- Changes made by the AI
- Changes made manually by the user
This would allow the AI to maintain a persistent understanding of the work rather than repeatedly reconstructing context.
Human remains in control
The system should support different autonomy levels:
Observe
- AI watches and understands
- Does not change anything
Suggest
- AI proposes changes
- User approves them
Assist
- AI performs low-risk actions
- User can immediately undo them
Agent
- AI performs multi-step tasks autonomously
- User can interrupt at any time
A persistent visual history could show exactly what the AI changed and provide one-click undo/rollback.
Example: creating an animation
Imagine a user creating a short animation.
They work normally while talking to the AI:
«“Move the character slightly to the left.”»
AI performs the change.
«“The movement feels too robotic.”»
AI analyzes the animation and suggests:
«“I can smooth the acceleration and deceleration.”»
User:
«“Do it.”»
AI modifies the keyframes.
The user then points with a pen at a specific moment in the timeline:
«“The problem starts here.”»
AI understands the spatial reference and adjusts that portion.
The user never has to stop working to formulate a technical description of the software’s internal structure.
This could become a general AI operating environment
The long-term vision could be broader than a ChatGPT feature.
It could become an AI-native workspace layer between the user and their computer.
The user could say:
«“Build me a dashboard for this project.”»
The AI creates it.
The user points to an area:
«“Put this on the right.”»
The AI moves it.
The user switches to another application:
«“Take the numbers from this spreadsheet and update the graph.”»
The AI performs the cross-application task.
The user opens another monitor:
«“Use this monitor for the research sources.”»
The AI reorganizes the workspace.
The computer effectively becomes a shared physical workspace for the human and the agent.
Hardware should not be a requirement
The concept should work at multiple levels:
1. Laptop + ChatGPT
2. Single monitor + split ChatGPT workspace
3. Two or more monitors
4. Phone + monitor
5. Tablet + external display
6. Touchscreen + pen
7. AR/VR displays in the future
The phone is particularly interesting because it could become a natural dedicated AI controller:
Voice + Chat + Camera + Authentication + Remote control
while the larger screen remains dedicated to the actual work.
Potential interaction model
A persistent AI workspace could have four simultaneous channels:
Voice
“Make this smaller.”
Text
Detailed instructions, planning and technical communication.
Visual
The AI sees the workspace and understands its state.
Spatial
The user points, draws, circles, taps or drags to identify exactly what they mean.
The combination of these four channels is much more powerful than any individual modality.
The larger opportunity
I believe the fundamental opportunity is not simply “AI controlling a computer.”
It is creating a shared workspace in which the human and AI continuously perceive, discuss and manipulate the same environment.
The user should not have to constantly switch between:
«“My work”»
and
«“ChatGPT”»
Instead, ChatGPT should become part of the working environment itself.
The ultimate goal would be:
Don’t make the user explain the workspace to the AI. Let the AI enter the workspace with the user.
I believe this could be particularly powerful as AI agents become increasingly capable of operating software, creating content and executing multi-step tasks.
Thank you for considering the idea.
Best regards,
Ghavam