For a given conversation in ChatGPT, could one add time context to the models. If the user inputs a prompt directly after another response vs the next day, the context of what they are saying in relation to the previous response is less likely of being important. In a distant future, maybe one day chat gpt could monitor basic operations of one’s computer, such as “start powerpoint”, or “opened new tab”, or what the user searched, and other low information inputs. This would give more context to any question asked by the user. Essentially some sort of low information tracking, of activities and processes on the computer… But one would have to sign away their privacy… One nice thing is that most of this could be done in an app, where the app uses the computers resources to process what is going on the laptop, and then sends chat gpt a very stripped down low information input of the key actions and processes going on for the computer. It could even be a screen recording process, where it takes a frame every 0.3 seconds, and identifies and lists the text on the screen and sends chat a binary image of text vs background. The point is, right now chat gpt is bound by the web, it would be nice if the users computer could take part of the workload! Thoughts?
I think you just described Windows Recall.
In addition to Windows Recall, you might find the UFO project to be interesting.
But one would have to sign away their privacy…
A user’s task context (which tasks that a user are performing and where they are in those tasks) could resemble a powerful feature where a user would provide a sort of permission per external agent or website seeking to access it.
Users might also be able to simply pause and resume the management of their task contexts.
With users’ task context data available, developers would be able to provide next-generation context-aware and adaptive solutions, these including: adaptive hypermedia, context-sensitive user interfaces, context-sensitive help and documentation, search engines, question-answering systems, recommender systems, and AI assistants.
With users’ task context data available, developers would be able to provide next-generation automatic attention management features. Users wouldn’t, then, be interrupted in the middle of their tasks by non-urgent messages, notifications, or alerts.
In a proposal sketch which I wrote for discussion purposes, I modelled users’ task contexts resembling “business process model call stacks”.
There, I presented a thought experiment with a client-side context management application – described in some detail – which users would have to interact with while they performed their tasks to maintain their task contexts.
Obviously, a goal of the thought experiment is to consider how client-side AI systems, including by making observations, could automatically manage users’ task contexts for them so that they could focus the entirety of their attention on their tasks. For example, what sort of API would a context management application need to provide? Should client-side AI systems be able to manage users’ task context data for them automatically, context management applications would be able to become background processes or services…
Interesting, contemporary, and complex topics! What do you think of these preliminary ideas?
In addition to enhancing AI systems’ awareness of what users were doing to provide them with benefits and features, there is also enhancing AI systems’ awareness of what they, AI systems, were doing.
With some suitable means of representing task contexts, in theory, users or AI agents could ask multi-agent systems, e.g., via interface agents, about the tasks that the systems were performing and where they were in these tasks.
Perhaps some agentic systems could examine and discuss their own workflows and the current situations in these…
These kinds of capabilities might be useful for scenarios where AI systems were desired to engage in and to complete longer tasks, e.g., tasks with durations measurable in hours or days. Users would be able to chat with these systems about their tasks, subtasks, progresses, and, perhaps, estimated times remaining until completion.
Additionally possible is that other client-side applications could synchronize from AI systems’ task context data to provide users with interactive visualizations and analytics.