ChatGPT Breaks into the Physical World: Introducing Shifu Plugin for ChatGPT

As a developer passionate about the potential of AI and IoT devices, I was thrilled when I got early access to the ChatGPT plugin. Inspired by ChatGPT’s capabilities and my knowledge of IoT devices, I embarked on a journey to create a plugin that enables ChatGPT to interact with and control the physical world.

The Vision for an AI-Connected World

My fascination with ChatGPT led me to envision a new way of connecting AI with the physical world. By enabling AI to access real-world data through IoT devices, I believed we could create more engaging and meaningful experiences for users. With the right tools and infrastructure, I knew that bridging this gap was possible.

ChatGPT + IoT = Shifu Plugin for ChatGPT

Fueled by my passion for AI-human interaction, I set out to create a plugin called “Shifu Plugin for ChatGPT” that would integrate ChatGPT with IoT devices. This plugin allows ChatGPT to not only extract data from devices but also control them. For example, by connecting an IP camera through the plugin, users can display real-time images and control the camera within chat conversations, opening up new possibilities for AI-human interactions. I will open source the plugin very soon. Please star the repo and stay tuned: GitHub - Edgenesis/shifu-plugin-for-chatgpt: A plugin helps ChatGPT to interact with IoT devices.

The Power Behind the Plugin: Shifu

To create the Shifu Plugin for ChatGPT, I utilized an open-source infrastructure called Shifu: GitHub - Edgenesis/shifu: Kubernetes-native Industrial Edge. Shifu is a Kubernetes-native industrial edge that allows complete IoT interoperability by virtualizing devices into Kubernetes pods. This adaptability and flexibility made it possible for seamless integration with ChatGPT, providing AI access to and control over the physical world. This approach can be applied to a wide range of IoT devices, further expanding AI-human interaction possibilities.

Emboiment: Taking a Step Towards AGI

One of the essential aspects of the pursuit of Artificial General Intelligence (AGI) is AI’s ability to interact with the physical world. By developing the Shifu Plugin for ChatGPT, which connects ChatGPT with IoT devices, I have enabled AI to access and control the environment. This integration is a significant step forward in the quest for AGI, as AI-human interaction can extend beyond the digital realm.


Developing the ChatGPT plugin has been an incredibly rewarding experience, showcasing the potential of collaboration between AI, IoT, and developers. By connecting AI with the physical world using the Shifu Plugin for ChatGPT and Shifu’s infrastructure, we can pave the way for more engaging and meaningful AI-human interactions. I am excited to see the continued advancements in AI as we explore new ways to bridge the gap between the digital and physical realms, and I am proud to be part of this innovative journey.


this is amazing. but also I am so afraid of giving chatgpt access to real world systems tho hahaha

This is how an AGI could actually do damage. IE connect to heating system and turn heat way up or something. Amazing but terrifying on the other side.

On a cool positive note.
I would love to tell chat gpt to turn off all the lights in my house, or play songs on my IoT speaker systems.

Im both excited and terrified to see where this goes!

Best of luck.

1 Like

True. We need a baymax, not a skynet :wink: