Introducing MacCopilot: interact to your screen with GPT-4o freely

Hi there

Just found so many interesting ideas on OpenAI community.:blush:

Like many others, I’ve been thinking about the better way integrating AI with our daily work. Thinking about how we can best utilizing the power of LLM like GPT-4o.
A couple month later, may be a year, after being use ChatGPT and GPT-4 API, and some other third-party Copilot AI - like client in my daily work. I started to realize that how natural it is when you can talk to your working space(the screen) directly with AI anytime, with just a screenshot taking. I mean visually talking to, not copy-pasting text.

Along with the recent release of GPT-4o, which beat GPT-4 with more powerful multimodal capability and lower price. I think it is time to create such a copilot app. That’s how MacCopilot come from.

How to use

  1. Take screenshot by keyboard shortcut;
  2. Input the question.

That’s it, thanks to GPT-4o, you can talk to your whole screen.

Show case

It is a paid app with no free tier for now. You also need a OpenAI API key that can use GPT-4o model to use it.

You can visit maccopilot.co for more information, or buy it on Gumroad - MacCopilot.

Other features

Beside gpt-4o, gpt-4-turbo, gpt-4-vision, MacCopilot also support other platform like Claude Opus, Gemini AI. Azure OpenAI.

Plans

  1. A ready-to-use subscription with builtin API key.
  2. Integrate open-source multimodal models.

Hope it can help others think about more forms of Copilot AI.

Please feel free to share thought below.