Inconsistent Behavior When ChatGPT Reads Uploaded Zip Files for Codebase Analysis

GPT builders
,
1

Hi everyone,

I’m working on a proof-of-concept (POC) to use a custom-configured ChatGPT for generating test cases and Cypress automation scripts based on structured requirements. The setup involves uploading a project-specific .zip file containing Cypress tests, commands, and helper functions, with the goal of having ChatGPT analyze and utilize this codebase in its responses. However, I’ve encountered some issues with inconsistent behavior when ChatGPT attempts to read and integrate the uploaded files.

Issue Summary:

  • Inconsistent File Recognition: Often, ChatGPT does not recognize the uploaded .zip file on the first attempt. Occasionally, it requires a follow-up prompt, such as “Have you read the zip file?” to trigger it to acknowledge the file.
  • Inconsistent Use of Existing Code: Even when ChatGPT successfully unpacks the file, it does not consistently utilize the existing commands and helper functions in the way specified in the prompt. This inconsistency becomes especially apparent in new sessions, where prior helper functions are not reliably reused, and ChatGPT may re-interpret or rewrite helpers differently.
  • Memory Continuity Gaps: Since ChatGPT does not retain session-based memory, each session starts fresh without retaining any helper functions or commands created in prior sessions. This requires re-uploading and retraining, often with inconsistent results.

Desired Behavior:

Ideally, I would like ChatGPT to:

  1. Reliably recognize and analyze uploaded .zip files without additional prompts.
  2. Consistently reuse existing helper functions and commands within the same session and across sessions, applying the same approach each time it generates code.

Questions for the Community:

  • Has anyone else experienced issues with ChatGPT inconsistently unpacking and reading .zip files? Are there any best practices to ensure it reads the contents on the first attempt?
  • Any recommendations on achieving more consistency in how ChatGPT applies existing code, especially in cases where session continuity is limited?
  • Are there alternative approaches to handling helper functions across sessions to improve consistency?

Appreciate any insights or advice on overcoming these limitations!

1 Like