Can Custom GPT be Adapted to Python Code for Documentation and Analysis?

ChatGPT
, ,
1

Hello everyone,

I’m exploring the possibility of using Custom GPT models to specialize in understanding and working with a specific Python code base. My goal is to upload a Python script to the Custom GPT so that it can learn the script in detail. Ideally, this would enable the GPT to:

  1. Write comprehensive documentation about the script’s usage
  2. .Identify weaknesses or areas for improvement within the script.
  3. Suggest enhancements to evolve the script into a more complex and robust system.

To test this, I experimented with a relatively small Python script (approximately 300 lines). After uploading it to the Custom GPT, I asked the model to identify the functions present in the script. Unfortunately, the results were disappointing. The model mentioned several functions that do not exist in my script, indicating some sort of hallucinations or something else altogether.

Before proceeding further, I would like to gather insights from you guys:

  • Has anyone had similar experiences with Custom GPT models?
  • Are there known limitations or best practices for training Custom GPT on specific code bases?
  • Could the model’s inability to accurately reflect my script’s functions be due to the size or complexity of the script, or perhaps an issue in how I set up the Custom GPT?

Any advice, experiences, or pointers you could share would be greatly appreciated. I’m keen on making this work and am open to suggestions on how to better train or utilize Custom GPT for this purpose.