Metaprogramming with Codex!

Prompting
1

Hello OpenAI Community!

Intro:

Metaprogramming has always been a favorite topic of mine, since it enables users to do various tasks such as code-generation and manipulating code during execution to help keep code D.R.Y (Don’t Repeat Yourself). Utilizing metaprogramming correctly can allow you to create very powerful systems.

The use cases for metaprogramming extend far beyond what I’m showing here in this post, but I hope to show the core functionality of it here to show how powerful it can be as well as hopefully inspiring some people along the way!

What is Metaprogramming?

For those of you who have not heard of metaprogramming and would like a simple example, here is a screenshot from a topic discussing metaprogramming using Python by Real Python:

Python_metaprogramming_example
Python_metaprogramming_example692×533 7.11 KB

Essentially, when you create any object in Python, the root class of the object is of class ‘type’. Here’s a function I spun up that shows you what I mean:

def inspect_object_type(object_being_inspected):
    object_type = type(object_being_inspected)
    iteration = 0
    # Print the object type until it equals itself (the root class in Python)
    while(object_type is not type(object_type)):
        print(iteration, object_type)
        object_type = type(object_type)
        iteration+=1
    print(iteration, object_type)

simple_str = "Hello World"
inspect_object_type(simple_str) 

#0 <class 'str'>
#1 <class 'type'> <-Last iteration always is of class 'type'

With that being said, I wanted to dive into how Codex would perform when tasked with creating a metaclass to see if it could:

  1. Correctly define a metaclass
  2. Use the metaclass as directed within other classes

Here’s how Codex performed:

Prompt:

"
You are a programmer that is tasked with creating a basic metaclass.

Tasks:
Create a metaclass named ‘Meta’. In the static method ‘new’, add the parameters ‘clsname’, ‘bases’, ‘clsdict’. Inside the ‘new’ static method, add a print statement that says “This should run first”
Create three classes with the metaclass parameter set to the metaclass you created. Initialize each of them with a string variable that is set to their unique class names
“”"

Completion:

Davinci-Codex_Metaprogramming
Davinci-Codex_Metaprogramming1361×738 31.7 KB

Settings:

Engine: davinci-codex
Temperature: 0
Top P: 1
Frequency Penalty: 0
Best of: 1

Code Output:

This should run first
This should run first
This should run first
a
b
c

Conclusion:

The davinci-codex engine flawlessly tackles defining a proper metaclass and using it within other classes as instructed.

Knowing this, I’ll be exploring how the model does when tasked with creating more advanced metaclasses in order to see how Codex can help users with writing code without them having to work as hard.

Thank you OpenAI!

Interested in metaprogramming? Try out my preset Codex Metaprogramming on OpenAI’s Playground!

Inspired by David Beazly’s talk on Python 3 metaprogramming

openai_twitter_logoHelpfulDev

10 Likes
2

this is awesome, thank you for exploring this space!

4 Likes
3

Now gotta find the cases it does not work.
How about importing an existing code and helping the user modify it ?

1 Like
4

That’s a good idea, @renanm.b! I do plan to do that with Codex. I’d like to work my way up in terms of complexity when working with Codex so that I get better at prompt engineering and ensuring that my instructions are as clear as possible for Codex so that the desired output is cleanly interpreted.

One other area I am going to explore is test-driven development, that is, writing tests in Python that are commented for clarity and seeing if Codex can write a script that passes those tests. After that, I’ll be exploring your idea :grin:

1 Like
5

you can use BERT and a basic reinforced learning with self-evaluation.
It is 90% similar to OpenAI Gym problems.
It makes the completions better.

2 Likes