CreatiCode Scratch Plugin - Enable ChatGPT for Block-Based Coding

Hi everyone, we are proud to present our plugin “CreatiCode Scratch” to the community. This plugin helps ChagGPT to display block-based programs as images and learn to use new blocks in the CreatiCode extensions.

image

MIT Scratch is the de-facto first programming language for 2D animations for almost every K-12 student today around the world. The CreatiCode platform inherits the simple block syntax and web-based project sharing from MIT Scratch, and further allows students to explore many cool technologies that make learning to code fun and meaningful, such as 3D game engine (based on Babylon.js), Artificial Intelligence (ChatGPT, DALL-E, Speech Recognition, Body/Hand Detection, etc) and Augmented Reality, etc.

Since ChatGPT does not know how to display block-based programs, it usually uses pseudocode to represent Scratch programs. This plugin helps ChagGPT convert these programs into images, which are much easier to read for Scratch leaners and teachers.

In addition, ChatGPT can query this plugin to “learn” how to use newly defined blocks on the spot, and then make use of them to write programs or answer questions.

We expect this plugin to become a great teaching/learning aid for K-12 STEM teachers and students. Here are some example prompts to try it out:

  1. Draw a star with pen

image

 
2. Draw a colorful rose with ovals

 
3. Create a 3D table

 
4. create a chat app using chatgpt

 
5. show me how to use clone id

image

 
6. show me an example of using clones

 
 

Upcoming Features:

  1. Link to the project in the CreatiCode playground, so you can start running or changing the program generated by ChatGPT
  2. Handling more complex requests
12 Likes

Congrats on the launch!

How did the process go for you? (About to dip my toes into the plug-in pond…)

1 Like

One of the most required and very important plugin for juniors to get involved in software development early. Well done!

1 Like

Thanks. The overall process is mostly about writing a very big prompt (if you already have the API).

1 Like

Love your plugin! It looks really clever. How did you get ChatGPT to generate your blocks?

@mollusk Thanks! We have to teach ChatGPT the syntax for writing Scratch programs. It already knows some but its code is not always consistent.

Is there a standard textual syntax for Scratch programs? sb3 files are binary.

And isn’t it the case that ChatGPT has seen millions of Python and JavaScript programs but nowhere near as many Scratch programs? Or if all ChatGPT knows about the Scratch syntax is from the prompt is that enough to come close to the quality of Python or JavaScript programs that ChatGPT can generate?

There is no standard syntax for Scratch, so when ChatGPT generates pseudocode for Scratch it can use different syntax at different times.

As you said, ChatGPT can’t parse any Scratch project directly, so we believe it has only seen pseudocode on other websites that other people write to represent Scratch. But that should be enough for it to understand how it works.

On top of that, CreatiCode Scratch introduces many new tools, such as blocks for debugging, 3D, AI and AR projects. These are mostly published after the Sep 2021 cutoff, so we have to teach ChatGPT how to use them.

Where is the download? The button is not working.

Hi there, we are still working on generating a new project link so that you can run it in our playground at https://play.creaticode.com/ directly. Please stay tuned!

1 Like

So how can i get it to work with chat GPT? How do i get to the plugin

Hi @Salmaa

We have now integrated ChatGPT in our coding playground (creaticode.com) as a coding copilot named “CreatiCode XO”, which can explain/debug your program and assist in creating new projects. It’s located at the bottom right corner of the coding playground:

image

I just tried to sign up with on your site as a Teacher (I’m teaching kids how to code) and also trying your plugin but the request is taking a long time. Do I need to be approved first for the plugin to work?

It’s been there for 5 minutes or so.

Hi,

We need some time to verify teacher accounts. Sorry about the delay. It is approved now.

1 Like

Hi there, thanks for adding me but I believe I used a different email from my ChatGPT account email to sign up for a teacher’s account. It works now anyways but I noticed the click green flag block is a different color.

yours:

Scratch’s:

image

It’s not a major issue at all just an fyi. Thanks for this, it’s going to save me so much time as I came up with a good workflow that is like:

Yes, that’s a great approach. You can use the visual diagram of the Scratch program I provided (the cat moving in a square and meowing at each corner) to recreate the program in your Scratch GUI. Once you’ve recreated it, you can then export the project as a JSON file. Here’s how you can do it:

  1. Recreate the Program in Scratch GUI:

    • Open your local Scratch GUI.
    • Manually recreate the program by dragging and dropping the blocks as they appear in the visual diagram.
  2. Export the Program as JSON:

    • Once you’ve recreated the program, you can save it.
    • In the Scratch GUI, there should be an option to save or download the project. This option typically allows you to save the project as a .sb3 file, which is essentially a ZIP archive containing the JSON representation of your project and any assets it uses.
  3. Extract JSON from .sb3 File:

    • Rename the downloaded .sb3 file to a .zip file.
    • Extract this ZIP file. Inside, you’ll find a file named project.json. This file is the JSON representation of your Scratch project.
  4. Load the JSON When Needed:

    • You can now use this project.json file to load the program whenever needed.
    • If you want to share this with your students, they can import this JSON file into their Scratch GUI to view and interact with the program.

This method allows you to prepare Scratch programs in advance and easily distribute them to your students for instructional purposes.

hi @nyck33

Yes, we updated the color of the event blocks to dark red because many teachers find the original light yellow color makes the text on them hard to see on projector screens. The red color provides a much better contrast.

1 Like

do you have any plans for an api? Ie. user sends images and json of the scratch program to be analyzed and then gets a response back.

@nyck33

That’s an interesting idea. We haven’t got any requests like that since most of our users are teachers and students. We can definitely try to accommodate that. Please email to info@creaticode.com to discuss your use case further.

Thanks

I could not continue a conversation I was having with Creaticode Plugin and was directed to this page.

Are there plans for a Custom GPT?

Hi @nyck33

Unfortunately, our plugin will be discontinued per OpenAI. Our current plan is to provide this service on our platform at CreatiCode.com. If there is some specific task that you are looking for, please let us know, and we will make sure our AI Copilot can provide a similar service.