GPT-3 + Lego MindStorms

Hi everyone,

Glad to be a part of this community! Lots of creativity in here! :slight_smile:

While currently going through the documentation and doing some research it suddenly got me thinking about a use-case involving robotics.

Would it be possible to link GPT-3 with e.g. a Lego MindStorms robot? To give you an idea: MindStorm’s Python API allows you to control the entire robot such as its movement, color sensor and distance sensor.

There’s quite a lot of examples / documentation on Python programming for MindStorms as well.

https://github.com/arturomoncadatorres/lego-mindstorms

Did anyone try something similar, and / or is this allowed within the API guidelines (for personal research)? And do you guys think you would be able to train GPT-3 to interact with the robot?

Would be interested to hear your thoughts :slight_smile:

1 Like

This sounds amazing and ridiculously fun. You’re thinking GPT-3 could control the robot? I’ll be curious to hear what others say here.

Well I’m actually tempted to get one just to try it out, lol. First want to get myself more familiar with the API and its capabilities. I haven’t really put too much thought into it yet, but I suppose these scenarios seem pretty reasonable to implement:

  • Communicating to you by using LED or sound (e.g. responding by flashing its color red or green when something is true or false).
  • Training it to respond to moving instructions by giving it a couple of examples.
  • When it is about to collide with an object, teaching it to stop automatically or change its directions to a safer location (don’t think you’d really need GPT-3 for this, but it would be cool to compare it to a hard-coded implementation).
1 Like

Hy falko,
I hope you are doing well. And I wonder if you could get started with some development or improved ideas?
I have a Lego Mindstorms at home.
But, honestly, I have never tried to install and run Python code withing the blocks. I am a good learner, well!!’
So I wish to try and get this project going and updated.
Best regards,
Paulo Magrani