Feature Request - "Computer Use" via Timestamped Video, with Timestamped Key Logging and Cursor Position

ChatGPT Feature requests
1

The title explains it already.

The current implementation of “Computer Use” is incredibly primitive at best - more like a demo or proof of concept than any kind of functional feature.

Most actions and activities take place in a sequence of time, with all kinds of interactions and visuals that are not available from a static screenshot format. Furthermore its not really capable of “Using” anything in this way - except perhaps a simple webpage.

The model can recognize Video input natively - it already has this ability. It should not be very difficult to implement a “video capture/viewer” which runs a Timestamped chronological log, along with a Timestamped log of Keyboard Inputs and Cursor Positions with Mouse Clicks.

This would be an actual and true situation of “Computer Use” that would be actually valuable in terms of building and error checking.

3

Hey @Invisible_Acropolis! Appreciate you for sharing this.

I understand the request here. You’re asking for Computer Use to work with a more complete timeline of activity, rather than only relying on static screenshots.

A video-style viewer with timestamped screen changes, cursor movement, clicks, and keyboard input could make it much easier to follow what happened during a workflow, especially for debugging, QA, and error checking.

That makes sense as feedback, since a lot of computer tasks depend on sequence and timing, and that context can be hard to capture from individual screenshots alone.

This would likely need careful privacy and security controls, since screen recordings and input logs could include sensitive information. I can’t share a timeline right now, but I’ll pass this feedback along internally.

- Sunny