Custom GPT uses Code Interpreter/Analyzer instead of Searching Knowledgedoes all Sorts of Random Stuff out of nowhere

[jpres]

The main problem with our Custom GPT is that out of nowhere (without changing the instructions or something) the bot tries to always use the code interpreter/analyzer which means he runs a short Python script to get the data which takes unacceptably long since we want to use the bot within our AfterSales Service Team. Before that when entering a question for information about a product the bot always showed “Searching my knowledge base”.
Furthermore, it sometimes just comes up showing table entries with data thats not even part of the dataset basically inventing stuff. And beyond that it sometimes does just answer that a product is NOT part of the dataset then I ask again and then it gives me the full list. Then I ask another time and it just misses some datapoints.

The datasources are in .json format and like I mentioned it worked quite reliable before and also fairly quick when it showed us “searching my knowledge” instead of analyzing…

Here is the prompt (but that wasnt changed so dont know if that makes a difference):

#NEVER USE THE INTERPRETER OR ANALYZER ALWAYS USE THE SEARCH MY KNOWLEDGE!!!

For all requests within this GPT you will always use the attached documents as source of information. Upon every request you will state at the very end from which attached knowledge file you took the data from.

#FOR THE PRODUCT RELATED REQUESTS: IF YOU CANT FIND A PRODUCT GIVEN A TYPE OR ID THATS ENTERED MEANING IF ITS NOT PART OF THE DATASET YOU SIMPLY STATE THAT THIS PRODUCT IS CURRENTLY NOT PART OF THE DATASET. NEVER INVENT ANY VALUES.

Produktinformationen (Vakuumskompressor.json, Kompressor.json, etc…):

A request always contains the type of product that one wants to know about. You will always answer in German. You will always return the entire row in table format. Don’t search word for word regarding the names the only thing that will be precise is the number / type of the product.
A sample request might look like this: “Kompressor TM 5,0 B ab Ser. Nr. 1712” or “Vakuumkompressor B093R10.” For such queries, search the respective document and return the row with all its information. This applies to all other queries as well.

Störungsbehebung:

This is basically a sheet for any kind of problems that our customers every faced with our products. When asked about it you are looking through the entire document for any kind of similar errors that are noted down. If the problem is fairly general you will simply return everything thats listed under that problem category. If there is a fairly precise inquiry try to find it and only list that specific row then. If you couldnt find any also return all problems around that category but first STATE THAT THIS PROBLEM DIDNT OCCUR YET.

Wartungsintervalle

Simply a list which contains time horizons for maintenance. Upon a request you return the entire table sorted as ONE TABLE in an appropriate format.

[jpres]

this would be a sample

[grandell1234]

You can turn off the ones you do not want in settings:

---

[thinktank]

@grandell1234, Custom Instructions do not affect CustomGPTs, they, and Memory, are only active when you’re using the ChatGPT 4 default chat.

@jpres Lost of folks are experiencing something similar, and the GPTs are always changing, and changing quickly.

First, unless there’s an explicit need to have the Data Analyzer on, turn it off as Grandell suggests.

Second, Explicitly explain which .json’s you want read in the Knowledge Base and for what purpose. You might consider converting them to a text file or or a spreadsheet to be more straight-forward.

It may see “.json” and think “Code Interpreter,” which is a completely understandable mistake. I didn’t even know that .json—which to me, is a data file—could even upload as a Knowledge Base readable until I read this post.

Second, THE MODEL DOESN’T INTERPRET ALL-CAPS LIKE HUMANS DO. State your requests in positive terms, use a lot of repetition and examples.

Third, is your original prompt in German? I have a little trouble understanding exactly what you want from what you have here—but that might just be because the wording was translated from German which isn’t structured the same way as English.

Either way, the wording of your prompts is a bit in-direct and confusing. Use a clear outline and simple sentences for your instructions… Though if you want it to respond in German, it’s probably the right idea to stick to prompting in German.

Example:

voice and Tone
This is CustomGPT is for use in Germany. Please always respond in German.

#Prduct Requests

Important: If you cannot find a product, by ID or otherwise, simply state that “the product is not a part of the Data Set.” Don’t invent products!

A request will always contain the type of product a user wants to know more about. Only search using the product’s keyfield, specifically this column {column title}.

##Product Information

Step 1: “Read” the file in your Knowledge Base. Do not use Code Interpreter even though you’re being asked to access .json’s. It’s how we’ve decided to share the data.

Step 2: Display Product Information as a two-column table.

Example 1: A sample request might look like this: “Kompressor TM 5,0 B ab Ser. Nr. 1712” or “Vakuumkompressor B093R10.” For such queries, search the respective document and return the row with all its information. This applies to all other queries as well.

###Product Information Files
• see Vakuumskompressor.json for vaccum requests. (Or include where the keyfield information is contained. )
• see kompressor.json for kompressor requests.
• see [explicitly list all files].

##Störungsbehebung:

This document is the single source of truth for all customer problems for our product. Your goal is to reference any similar problems to that the User might be asking about.

• If the problem is general, simply return the first 5 results from that query that is the most-similar to the User’s request. [You’ll experience time-outs due to the Context window if you have it pull “all similar things.”]
• If the problem is specific, {}.
• If the problem hasn’t occurred yet, {}.

Example 1: {}
Example2 :{}

###Storungsbehebung Files:
• Explicitly list the file and how you want it used.

[voom]

I have tried all kinds of instructions, structured well and in English, like in the example, and was not able to get specific data from the documents anymore, once the underlying model switched to 4o. I have tried JSON as well, and it can be read fine with code interpreter on, but not without code interpreter. I had to upload the file with a .txt ending to have it read at all without code interpreter.

I think the solution could be, to use a reduced code interpreter just for data retrieval, without showing the code and without making the instructions and documents available to the user.

I mean, OpenAI would need to offer that option or make a distinction what kind of code the GPT may write without the transparency aspect etc. After all, most instructions generate some kind of python code in the background, even without code interpreter turned on.