🖥 ChatPC - Mac automation meets AI

[dounan]

Update 1/13/24

With the launch of the GPT store, you can now use the ChatPC GPT! I’m so happy that OpenAI moved in this direction The ChatPC GPT has internet browsing, DALL-E image generation, and code interpreter capabilities turned on, so you can apply these capabilities to your local data (e.g. read a local CSV file and perform data analysis with code interpreter).

ChatGPT - ChatPC - connect with macOS

Update 10/26/23

ChatPC

Got a big update to share!

I’ve brought the plugin capabilities natively to ChatPC to leverage AI’s capabilities to make automation simpler, more intuitive, and more powerful.

ChatPC now allows you to use simple English to automate your Mac. It can manage files and interact with other Mac applications through Shortcuts and AppleScripts.

Check out this quick demo video

Safe by design - With ChatPC, you are in total control. By default, ChatPC cannot execute any actions on your Mac. You must explicitly give ChatPC access to the folders, Shortcuts, and AppleScripts that it can action on. In addition, you can configure which actions require your explicit approval before executing.

I’m currently looking for beta testers to try out the app and give me feedback. The app is free to use up to some token limit. If you hit the limit, DM me with your initial thoughts and the email address you signed up with – I’ll be giving free coupons to the first 100 beta testers!

To get started, check out the MacOS Getting Started Guide (MacOS 13.0 or higher required).

Update 9/4/23

ChatPC is pivoting to a standalone native mac chat experience and is no longer a plugin

Instead, ChatPC will focus on delivering the best native AI chat experience for the mac. Some key functionality:

• Global hot key to bring up AI chat anytime, in any app
• No more copy and paste. Bring selected text and files into your prompt with ease
• Inject the AI output to back into any application with a click of a button (or hot key)
• One click prompts to avoid typing the same things over and over
• Powered by GPT-4

And possible features down the line:

• Chat against local knowledgebases
• Chat against really long files
• And more…

Check out the new getting started page.

The plugin functionality will still be supported for the time being, but will no longer be actively developed. You can turn on the legacy plugin settings in the settings window

Update 6/9/23

DesktopGPT has been rebranded to ChatPC

Original post

Hi everyone!

I’m excited to share and get feedback on a plugin that I’ve been developing - ChatPC. It is like Zapier for your desktop, designed to allow safe and secure interaction between ChatGPT and your computer. It enables ChatGPT to read and modify local files, and interact with local applications while upholding strict access controls. You have the power to control precisely what actions ChatPC can perform and what data it can access.

Here are just some examples of what the plugin can do:

1. NEW Jul 12, 2023 - Summarize local files (even very long articles and entire books!). The example below summarizes a 9000 word article about the Twitter acquisition.
   

   

2. NEW Jul 7, 2023 - Question and answering on local files
   

3. Generate and open a new HTML file on your computer
   

4. Search your contacts and draft an email
   

5. Organize your files
   

6. Update code with tests and fix bugs
   

As of now, ChatPC is still in beta (not verified by OpenAI so only accessible to plugin developers) and exclusively available for MacOS users. However, based on the demand, there is potential for expansion to Windows and beyond.

To get started, follow the instructions in the MacOS getting started guide.

File access is built natively into the app, allowing you to select which folders ChatPC has access to. You can also set read and write permissions for each folder.

Question and answering on folder contents can be enabled with a click of a button:

Interacting with local applications is achieved through user added AppleScripts (and JXA). Rest assured, ChatPC can only execute scripts that you have explicitly approved and added.

IMO, the usefulness of this plugin is tied to the quality and availability of useful AppleScripts. While I’ve written a few, I am eager to see this collection grow with demand and with community contributions.

I am actively seeking feedback and suggestions for improvements. Don’t hesitate to share your experience or reach out with any questions. Your input can help shape ChatPC into a tool that serves us all efficiently. Thank you for your time and interest!

ChatPC - MacOS Getting Started Guide

[rkaplan]

Very nice

Do only Applescript scripts work or can you use JXA also?

[dounan]

Both AppleScript and JXA work!

I’ve updated my original post and getting started guide to also mention this

[rkaplan]

Thank you - sounds like a great project - I will give it a try

[dounan]

Awesome ! Please DM me or shoot an email to support@desktopgpt.ai with the email address that you signed up with so I can give you access to the beta. The Mac client won’t be able to connect to the server until that is done. Looking forward to hearing your feedback!

[jethro.adeniran]

I don’t have a MAC but this is exciting and I will be following it closely

[akram.osman]

Hey guys! Does anyone know when the awesome ChatPC plugin will finally be available for Windows? I can’t wait to connect ChatGPT to my local files and applications and level up my workflow. If anyone has any information, please share! Thanks in advance!

[jlailiftedup]

Great job, I was hoping to find something like this for a while now.

[cap]

It is Not a plugin but you can use with your document, is open source so you can add new functions: POWER-KI/GPT: GPT programs written in POWER-KI - Chat and PDF management (github.com)

[dounan]

@akram.osman

Hey guys! Does anyone know when the awesome ChatPC plugin will finally be available for Windows?

Hi Akram! Unfortunately I don’t have any concrete plans at the moment to support Windows to allow myself to focus on building out the Mac application. However I am keeping track of the number of requests for a Windows application and will keep you posted.

[zikass]

This is just impressive , the possiblities are endless and can’t wait for a windows plugin.
thank you for your work.

[jcanderson]

I have been dabbling with similar ideas to facilitate authoring other projects:

GitHub - jcaucb/AutoEditr: GPT Plugin that reads and modifies a specified folder

AutoEditr operates on a playground folder on the file system and performs CRUD operations on files and folders therein. I’ve found this to be really useful for working on multi-file documents and coding projects. It can invoke pytest on projects setting up a loop of being able to write and then test code and respond to error messages with new edits. I’ve played with letting ChatGPT write iterative improvements directly on the plugin’s code, mostly successfully. Actually, ChatGPT wrote the whole project, but through lots of prompting and copy & paste.

Some observations from working with it:

• The token limits really limit how much it can write. I’ve had it add append functions, line-range editing, and search and replace to try to get around this, but it’s continued to be a barrier. Even when ChatGPT could use these lower-token strategies, it tends to prefer rewriting entire files unless I remind it use alternatives.

• I’ve seen issues with it having trouble remembering paths and details of the content within a chat, requiring frequent re-reading of files

• It often goes crazy. I have to watch what it is doing and maintain backup files for when it starts rewriting the wrong thing, adding unwanted dependencies, etc.

• An outstanding question is how such a plugin can be deployed as a formal plugin – since it seems to inherently require access to the hard drive, how do you establish a single shared API for it?

[ARTHURONE]

When do you think you Mac version plugin will be available to everyone and will it woik with Intel models ??

[dounan]

When do you think you Mac version plugin will be available to everyone and will it woik with Intel models ??

This will work with both Intel and Apple Silicon models – it just requires MacOS 13 and above.

Regarding the plugin availability… I have some bad and good news.

The bad new is that I’ve been getting some pushback from OpenAI on getting this plugin approved because they say they cannot verify the safety of the Mac application that is required. I see other plugins that require the installation of Chrome extensions, which can also be malicious, so I’m not quite sure how they are verifying the safety of those extensions. I’m going to keep pushing here to see if they will change their policies.

The good news is that I am working on a few ideas to get around this limitation:

• I’m going to enable this to work as a standalone Mac application by providing a native chat application experience that will enable you to chat with GPT-4 and allow GPT-4 to interact with your local computer directly.
• I am thinking of other integrations where having an API into your local computer would be helpful – such as a Zappier integration (which can then also be accessed via the Zappier ChatGPT plugin)

I am also working on a few new exciting features that will be coming out soon:

• Question and answering on local files
• Summarization of local files
• Seamlessly inject the power of GPT-4 text generation into other native applications

Subscribe here to stay up to date! Or install the app and enable automatic updates – I’ll be including release notes to describe new features as they are released.

[ARTHURONE]

Thank you Dounan please keep work on it do not be discouraged allowing GPT-4 users to simply access their own hard drive folders and files will be a major advancement for many of us to gain very useful functionality. And since we are accessing our own hard drives no one else’s it is a very logical option of choice we should have. Thanks !

[dounan]

v0.9 has been released with question and answering on local files!

I’m happy to introduce a new feature that lets you turn your local folders into accessible knowledge bases. This allows you to interact with your local data more intuitively through question and answering.

For example, imagine you have a local folder titled “AI Research Papers” filled with up-to-date AI papers. After you’ve added this folder via ChatPC settings and switched on its question and answer capability, you could prompt ChatGPT like this: “ask AI papers: what is GPT-4?”.

Enabling question and answering is as simple as a click of a button:

[dounan]

Introducing large file summarization in v0.10!

You can now summarize local files, even extremely long articles or entire books that normally don’t fit into ChatGPT’s context limit!

Here is an example summarizing a 9000 word article on the acquisition of twitter:

While this feature does not require the folder to be indexed for question and answering, it is very useful to be able to ask follow up questions on the summary. For that reason, you may find it useful to also enable question and answering on the folder.

[Jackkk]

where to install the unverified plugin?

[Foxalabs]

Have you applied for and been granted developer plugin access? if not the waitlist is here

ChatGPT Plugins waitlist

[Jackkk]

I just fill the waitlist, how long would I wait from now?