Is anyone else finding Code Interpreter very unreliable? I’m finding really basic tasks like splitting information into separate text files impossible. The files generated have no content or partial content. It’s clear from the conversation that my prompts are being understood. I’m finding this happing more often than not.
Anyone else?
1 Like
Write the contrary, I’ve found it to be absolutely incredible lately—especially with the addition of custom instructions.
Because it seems after every time it runs code it begins a new message, it’s possible to get it to plow through almost an entire development process, start-to-finish, from a single prompt.
Basically you put some carefully written chain-of-thought prompt instructions in the Custom Instructions (hint: you can put instructions on how it should behave in the about_user_message, it will follow them anyway).
One thing I’m currently working through is basically a ghettoized information retrieval, self-reflection, and self-prompting system where the model churns through an entire development process.
This involves (very broad strokes),
- Including an instruction to pseudocode the project as a nested numbered list first, including the inputs and returns of all functions and write that to a text file,
- Then I have it create individual
.pyfiles for each function, which includes the function skeleton containing commented pseudocode in the function body, along with a summary of everything it needs to know about other functions in order to successfully write the code. I’m toying with having it write unit-tests too. - Then, per the instructions it reads in the text of the
.pyfile at the start of each step which (ideally) has all the context the model needs for the small task. - Once everything is written and tested, it combines everything into appropriate directory structure, zips the directory, and gives a download link.
I am far from having it perfect, but I’ve been able to get 10’s of thousands of tokens generated from a single prompt.
2 Likes
This is equally awesome as it is scary.
Having reminders in each file has worked really well for me as well.
Gosh, this would be a fun competition. One prompt (+ instructions) to rule them all, in a game of pong. Or something equally silly.
I sometimes encounter similar issues with reading files completely. Checking the work of the code interpreter then reveals that only the first 500 or 1000 characters of the document have been read. In this case it helps to specify in the prompt that the document should be read in full and that it has X characters.
The code interpreter will then read the whole file and you are off to the races again.
I first noted this behavior with the last model update but it’s not so bad if you can work around it.
Hope this helps.
1 Like
This observation is somewhat supported when using CI inside the Android app. Of course inside the app you cannot select the model type but when starting a CI session on the web and then continuing this conversation in the app reveals several sequential messages from the model in a row.
I first thought it was a bug but it’s actually just the CI that I have come to appreciate so much.
Attached a screenshot with several messages from the model in a row where the model fixes the truncation error matching the topic.
2 Likes
Well done you!
And thank you for sharing this!
For some reason I haven’t yet looked at the JSON of a conversation which used Code Interpreter, but I’m pretty sure you’d see it reflected there as well.
Incidentally, I think this might be one reason why we are unlikely to see a Code Interpreter API Endpoint anytime soon—the potential of runaway token use…
1 Like
It is possible to get those extremely long replies with the standard model, too. Even though it’s more challenging to achieve I think.
GPT4 wrote me a complete architecture for my latest game in one go.
Hold on to those special prompts of yours because they surely are invaluable. But who am I telling that.
As @vb explained, tell it to itclude the entire doc.
use the prompt phrase ‘from the first page to the last page and do not change the wordings in the document apart from the specific instructions’.
I had a similar issue with Code interpreter, I gave it a simple task to rearrange my txt file in a particular format and while the phrase above got it to include the entire doc, it didn’t arrange them as I wanted even with several iterations of the prompt.
1 Like
As of this moment I am struggling to make CI read my files as a whole on the first attempt.
Currently I need to send a second request with " You didn’t read the files in full which is a shake because we have put hard work in to gather all the contents" and then the model apologizes and finally reads the files in full.
Let’s give your prompt a go!
On a side note: this behavior is surely a cost cutting effort that makes absolutely no sense considering that either the number of requests to the model go up until finally the whole file is read or people will be frustrated or mislead by the results. I’d say this should be escalated to product management.
How big are the files?
I’ve added some 2 meg CSV files okay… haven’t tested upper limits, though…
1 Like
edited: made a mistake and rewrote this post.
each file is less than 5kb.
First thing I tried was sending some bulk file information 
now I have them separated as much as possible sent as a zip with instructions. Mainly so that other people can explore them the same way that I’ve prepared
1 Like
My apologies, the files are actually less than 5kb each with about 3500 characters of content.
The files are tiny. Weirdly when I did something else to arrange csv files, it did it perfectly (after a couple of attempts).
After trying and failing with the suggestion from @idris today I was able to get CI to read the whole files using this prompt:
Use code similar to this template to read the files in full:
'# Displaying the full content of the “1a-Project Goal and Top-Level Action Plan.txt” file to fully understand the project goal and action plan
project_goal_and_action_plan_content = file_contents[“1a-Project Goal and Top-Level Action Plan.txt”]
project_goal_and_action_plan_content’
I sure hope this will help!
Great thanks. This was the original I used (with lots and lots of follow up attempts). How would I word it using your suggestion?
Here are 10 scripts (identifiable by the ten headings: useful verbs 1,making a booking, going to places, questions, sport, free time, enough, useful adjectives, where I live, household chores)
I want each section to be saved in a text file. The file name should be the same as the heading, e.g. making_a_booking.txt.
Useful verbs 1
[
{
“sentencesSpa”: [“Tengo que”, “Tengo que pedir algo”, “Tengo que pedir algo porque”, “Tengo que pedir algo porque tengo hambre.”],
“sentencesEng”: [“I have to”, “I have to order something”, “I have to order something because”, “I have to order something because I’m hungry.”]
},
{
“sentencesSpa”: [“Quiero”, “Quiero cambiarlo hoy”, “Quiero cambiarlo hoy porque es”, “Quiero cambiarlo hoy porque es demasiado grande.”],
“sentencesEng”: [“I want”, “I want to change it today”, “I want to change it today because it’s”, “I want to change it today because it’s too big.”]
},
{
“sentencesSpa”: [“Quiero”, “Quiero sacar algo”, “Quiero sacar algo de la nevera”, “Quiero sacar algo de la nevera porque”, “Quiero sacar algo de la nevera porque tengo sed.”],
“sentencesEng”: [“I want”, “I want to take out something”, “I want to take out something from the fridge”, “I want to take out something from the fridge because”, “I want to take out something from the fridge because I’m thirsty.”]
},
{
“sentencesSpa”: [“Lo necesito”, “Lo necesito ahora,”, “Lo necesito ahora, pero”, “Lo necesito ahora, pero lo he perdido.”],
“sentencesEng”: [“I need it”, “I need it now,”, “I need it now, but”, “I need it now, but I’ve lost it.”]
},
{
“sentencesSpa”: [“Tengo que”, “Tengo que arreglarlo”, “Tengo que arreglarlo porque”, “Tengo que arreglarlo porque está roto.”],
“sentencesEng”: [“I have to”, “I have to fix it”, “I have to fix it because”, “I have to fix it because it’s broken.”]
},
{
“sentencesSpa”: [“Es importante”, “Es importante terminarlo hoy”, “Es importante terminarlo hoy porque”, “Es importante terminarlo hoy porque lo necesito”, “Es importante terminarlo hoy porque lo necesito para mañana.”],
“sentencesEng”: [“It’s important”, “It’s important to finish it today”, “It’s important to finish it today because”, “It’s important to finish it today because I need it”, “It’s important to finish it today because I need it for tomorrow.”]
}
]Making a Booking
[
{
“sentencesSpa”: [“Quiero”, “Quiero reservar una mesa”, “Quiero reservar una mesa para cuatro personas”, “Quiero reservar una mesa para cuatro personas para esta noche.”],
“sentencesEng”: [“I want”, “I want to book a table”, “I want to book a table for four people”, “I want to book a table for four people for tonight.”]
},
{
“sentencesSpa”: [“Necesito alquilar”, “Necesito alquilar un coche familiar”, “Necesito alquilar un coche familiar para este lunes.”],
“sentencesEng”: [“I need to rent”, “I need to rent a family car”, “I need to rent a family car for this Monday.”]
},
{
“sentencesSpa”: [“Acabo de reservar”, “Acabo de reservar una habitación”, “Acabo de reservar una habitación para una semana.”],
“sentencesEng”: [“I’ve just booked”, “I’ve just booked a room”, “I’ve just booked a room for a week.”]
},
{
“sentencesSpa”: [“Quiero”, “Quiero reservar”, “Quiero reservar una clase de yoga”, “Quiero reservar una clase de yoga para principiantes.”],
“sentencesEng”: [“I want”, “I want to book”, “I want to book a yoga class”, “I want to book a yoga class for beginners.”]
},
{
“sentencesSpa”: [“Acabo de pedir”, “Acabo de pedir una habitación”, “Acabo de pedir una habitación con vistas al mar.”],
“sentencesEng”: [“I’ve just ordered”, “I’ve just ordered a room”, “I’ve just ordered a room with sea views.”]
}
]Going places
[
{
“sentencesSpa”: [“Esta tarde”, “Esta tarde tengo que ir”, “Esta tarde tengo que ir a la farmacia”, “Esta tarde tengo que ir a la farmacia para más paracetamol.”],
“sentencesEng”: [“This afternoon”, “This afternoon I have to go”, “This afternoon I have to go to the pharmacy”, “This afternoon I have to go to the pharmacy for more paracetamol.”]
},
{
“sentencesSpa”: [“Suelo”, “Suelo ir al colegio”, “Suelo ir al colegio todos los días”, “Suelo ir al colegio todos los días en autobús.”],
“sentencesEng”: [“I usually”, “I usually go to school”, “I usually go to school every day”, “I usually go to school every day by bus.”]
},
{
“sentencesSpa”: [“Me gusta”, “Me gusta ir al gimnasio”, “Me gusta ir al gimnasio los domingos”, “Me gusta ir al gimnasio los domingos porque no hay”, “Me gusta ir al gimnasio los domingos porque no hay mucha gente.”],
“sentencesEng”: [“I like”, “I like to go to the gym”, “I like to go to the gym on Sundays”, “I like to go to the gym on Sundays because there aren’t”, “I like to go to the gym on Sundays because there aren’t many people.”]
},
{
“sentencesSpa”: [“Hoy”, “Hoy tengo que ir”, “Hoy tengo que ir al supermercado”, “Hoy tengo que ir al supermercado antes de preparar”, “Hoy tengo que ir al supermercado antes de preparar la cena.”],
“sentencesEng”: [“Today”, “Today I have to go”, “Today I have to go to the supermarket”, “Today I have to go to the supermarket before preparing”, “Today I have to go to the supermarket before preparing dinner.”]
},
{
“sentencesSpa”: [“Me encanta”, “Me encanta ir de excursión”, “Me encanta ir de excursión al campo”, “Me encanta ir de excursión al campo los fines de semana.”],
“sentencesEng”: [“I love”, “I love to go on hikes”, “I love to go on hikes to the countryside”, “I love to go on hikes to the countryside on weekends.”]
}
]Questions
[
{
“sentencesSpa”: [“¿Cuándo”, “¿Cuándo quieres quedar”, “¿Cuándo quieres quedar para ir”, “¿Cuándo quieres quedar para ir al cine?”],
“sentencesEng”: [“When”, “When do you want to meet”, “When do you want to meet to go”, “When do you want to meet to go to the cinema?”]
},
{
“sentencesSpa”: [“¿Cómo”, “¿Cómo te gusta el café,”, “¿Cómo te gusta el café, solo o con leche?”],
“sentencesEng”: [“How”, “How do you like your coffee,”, “How do you like your coffee, black or with milk?”]
},
{
“sentencesSpa”: [“¿Cuánto tiempo”, “¿Cuánto tiempo necesitas”, “¿Cuánto tiempo necesitas para estar listo?”],
“sentencesEng”: [“How much time”, “How much time do you need”, “How much time do you need to be ready?”]
},
{
“sentencesSpa”: [“¿Por qué”, “¿Por qué no te gusta”, “¿Por qué no te gusta la película?”],
“sentencesEng”: [“Why”, “Why don’t you like”, “Why don’t you like the movie?”]
},
{
“sentencesSpa”: [“¿Por qué”, “¿Por qué necesitas”, “¿Por qué necesitas tanto papel?”],
“sentencesEng”: [“Why”, “Why do you need”, “Why do you need so much paper?”]
}
]Sport
[
{
“sentencesSpa”: [“Suelo jugar al tenis”, “Suelo jugar al tenis los sábados”, “Suelo jugar al tenis los sábados en el club.”],
“sentencesEng”: [“I usually play tennis”, “I usually play tennis on Saturdays”, “I usually play tennis on Saturdays at the club.”]
},
{
“sentencesSpa”: [“Intento nadar”, “Intento nadar en la piscina pública”, “Intento nadar en la piscina pública cuando”, “Intento nadar en la piscina pública cuando puedo.”],
“sentencesEng”: [“I try to swim”, “I try to swim in the public pool”, “I try to swim in the public pool when”, “I try to swim in the public pool when I can.”]
},
{
“sentencesSpa”: [“Juego al voleibol”, “Juego al voleibol con mis amigos”, “Juego al voleibol con mis amigos cuando vamos”, “Juego al voleibol con mis amigos cuando vamos a la playa.”],
“sentencesEng”: [“I play volleyball”, “I play volleyball with my friends”, “I play volleyball with my friends when we go”, “I play volleyball with my friends when we go to the beach.”]
},
{
“sentencesSpa”: [“Mi hermano”, “Mi hermano juega al baloncesto”, “Mi hermano juega al baloncesto en un equipo juvenil.”],
“sentencesEng”: [“My brother”, “My brother plays basketball”, “My brother plays basketball in a youth team.”]
},
{
“sentencesSpa”: [“Los domingos”, “Los domingos me encanta”, “Los domingos me encanta hacer footing”, “Los domingos me encanta hacer footing en el parque.”],
“sentencesEng”: [“On Sundays”, “On Sundays I love”, “On Sundays I love to jog”, “On Sundays I love to jog in the park.”]
},
{
“sentencesSpa”: [“Todos los veranos”, “Todos los veranos tomo clases de vela”, “Todos los veranos tomo clases de vela en el club náutico.”],
“sentencesEng”: [“Every summer”, “Every summer I take sailing lessons”, “Every summer I take sailing lessons at the yacht club.”]
}
]Free time
[
{
“sentencesSpa”: [“Este sábado”, “Este sábado quiero”, “Este sábado quiero ir al cine”, “Este sábado quiero ir al cine con amigos.”],
“sentencesEng”: [“This Saturday”, “This Saturday I want”, “This Saturday I want to go to the movies”, “This Saturday I want to go to the movies with friends.”]
},
{
“sentencesSpa”: [“Me gusta”, “Me gusta ir de compras”, “Me gusta ir de compras los domingos”, “Me gusta ir de compras los domingos porque hay”, “Me gusta ir de compras los domingos porque hay menos gente.”],
“sentencesEng”: [“I like”, “I like to go shopping”, “I like to go shopping on Sundays”, “I like to go shopping on Sundays because there’s”, “I like to go shopping on Sundays because there’s fewer people.”]
},
{
“sentencesSpa”: [“Me encanta”, “Me encanta tocar la guitarra”, “Me encanta tocar la guitarra después del trabajo.”],
“sentencesEng”: [“I love”, “I love playing the guitar”, “I love playing the guitar after work.”]
},
{
“sentencesSpa”: [“Los sábados”, “Los sábados me gusta”, “Los sábados me gusta quedar con amigos”, “Los sábados me gusta quedar con amigos para cenar.”],
“sentencesEng”: [“On Saturdays”, “On Saturdays I like”, “On Saturdays I like to meet with friends”, “On Saturdays I like to meet with friends for dinner.”]
},
{
“sentencesSpa”: [“Cuando tengo”, “Cuando tengo tiempo libre”, “Cuando tengo tiempo libre me relajo”, “Cuando tengo tiempo libre me relajo en la playa.”],
“sentencesEng”: [“When I have”, “When I have free time”, “When I have free time I relax”, “When I have free time I relax on the beach.”]
}
]enough
[
{
“sentencesSpa”: [“No hay”, “No hay suficiente harina”, “No hay suficiente harina para hacer”, “No hay suficiente harina para hacer otro pastel.”],
“sentencesEng”: [“There isn’t”, “There isn’t enough flour”, “There isn’t enough flour to make”, “There isn’t enough flour to make another cake.”]
},
{
“sentencesSpa”: [“Este abrigo”, “Este abrigo no es”, “Este abrigo no es lo suficientemente grueso”, “Este abrigo no es lo suficientemente grueso para el invierno.”],
“sentencesEng”: [“This coat”, “This coat isn’t”, “This coat isn’t thick enough”, “This coat isn’t thick enough for winter.”]
},
{
“sentencesSpa”: [“No tengo”, “No tengo suficiente dinero”, “No tengo suficiente dinero para comprarme”, “No tengo suficiente dinero para comprarme el último móvil.”],
“sentencesEng”: [“I don’t have”, “I don’t have enough money”, “I don’t have enough money to buy”, “I don’t have enough money to buy the latest mobile phone.”]
},
{
“sentencesSpa”: [“El aula”, “El aula no es”, “El aula no es lo suficientemente grande”, “El aula no es lo suficientemente grande para todos los alumnos.”],
“sentencesEng”: [“The classroom”, “The classroom isn’t”, “The classroom isn’t big enough”, “The classroom isn’t big enough for all the students.”]
},
{
“sentencesSpa”: [“No tengo”, “No tengo suficiente información”, “No tengo suficiente información para tomar”, “No tengo suficiente información para tomar una decisión.”],
“sentencesEng”: [“I don’t have”, “I don’t have enough information”, “I don’t have enough information to make”, “I don’t have enough information to make a decision.”]
}
]useful adjectives
[
{
“sentencesSpa”: [“Es”, “Es un restaurante muy bonito,”, “Es un restaurante muy bonito, para”, “Es un restaurante muy bonito, para una cena romántica.”],
“sentencesEng”: [“It’s”, “It’s a very nice restaurant,”, “It’s a very nice restaurant, for”, “It’s a very nice restaurant, for a romantic dinner.”]
},
{
“sentencesSpa”: [“Las cascadas”, “Las cascadas son impresionantes.”],
“sentencesEng”: [“The waterfalls”, “The waterfalls are impressive.”]
},
{
“sentencesSpa”: [“Ese programa”, “Ese programa es”, “Ese programa es muy entretenido.”],
“sentencesEng”: [“That program”, “That program is”, “That program is very entertaining.”]
},
{
“sentencesSpa”: [“La montaña rusa”, “La montaña rusa parece”, “La montaña rusa parece bastante”, “La montaña rusa parece bastante peligrosa.”],
“sentencesEng”: [“The roller coaster”, “The roller coaster seems”, “The roller coaster seems quite”, “The roller coaster seems quite dangerous.”]
},
{
“sentencesSpa”: [“Su casa”, “Su casa parece”, “Su casa parece muy acogedora.”],
“sentencesEng”: [“Their house”, “Their house seems”, “Their house seems very cosy.”]
}
]where I live
[
{
“sentencesSpa”: [“Vivo en Nueva York,”, “Vivo en Nueva York, en un piso pequeño”, “Vivo en Nueva York, en un piso pequeño cerca de Central Park.”],
“sentencesEng”: [“I live in New York,”, “I live in New York, in a small apartment”, “I live in New York, in a small apartment near Central Park.”]
},
{
“sentencesSpa”: [“Vivimos”, “Vivimos en una casa grande”, “Vivimos en una casa grande cerca de la playa.”],
“sentencesEng”: [“We live”, “We live in a big house”, “We live in a big house near the beach.”]
},
{
“sentencesSpa”: [“Mi sueño”, “Mi sueño es vivir”, “Mi sueño es vivir en una cabaña”, “Mi sueño es vivir en una cabaña en la montaña.”],
“sentencesEng”: [“My dream”, “My dream is to live”, “My dream is to live in a cabin”, “My dream is to live in a cabin in the mountain.”]
},
{
“sentencesSpa”: [“Mi piso en Barcelona”, “Mi piso en Barcelona está a sólo 10 minutos”, “Mi piso en Barcelona está a sólo 10 minutos a pie”, “Mi piso en Barcelona está a sólo 10 minutos a pie de la Sagrada Familia.”],
“sentencesEng”: [“My apartment in Barcelona”, “My apartment in Barcelona is just 10 minutes”, “My apartment in Barcelona is just 10 minutes away on foot”, “My apartment in Barcelona is just 10 minutes away on foot from the Sagrada Familia.”]
},
{
“sentencesSpa”: [“Quiero vivir”, “Quiero vivir en el campo”, “Quiero vivir en el campo porque”, “Quiero vivir en el campo porque la ciudad es demasiado ruidosa”, “Quiero vivir en el campo porque la ciudad es demasiado ruidosa para mí.”],
“sentencesEng”: [“I want to live”, “I want to live in the countryside”, “I want to live in the countryside because”, “I want to live in the countryside because the city is too noisy”, “I want to live in the countryside because the city is too noisy for me.”]
}
]household chores
[
{
“sentencesSpa”: [“Los lunes”, “Los lunes tengo que”, “Los lunes tengo que fregar los platos”, “Los lunes tengo que fregar los platos después de la cena familiar.”],
“sentencesEng”: [“On Mondays”, “On Mondays I have to”, “On Mondays I have to wash the dishes”, “On Mondays I have to wash the dishes after the family dinner.”]
},
{
“sentencesSpa”: [“Suelo hacer la compra”, “Suelo hacer la compra los miércoles”, “Suelo hacer la compra los miércoles después del trabajo.”],
“sentencesEng”: [“I usually do the shopping”, “I usually do the shopping on Wednesdays”, “I usually do the shopping on Wednesdays after work.”]
},
{
“sentencesSpa”: [“Primero”, “Primero tengo que barrer el suelo”, “Primero tengo que barrer el suelo antes de pasar el aspirador.”],
“sentencesEng”: [“First”, “First I have to sweep the floor”, “First I have to sweep the floor before using the vacuum cleaner.”]
},
{
“sentencesSpa”: [“Odio planchar,”, “Odio planchar, pero es necesario”, “Odio planchar, pero es necesario para la ropa de trabajo.”],
“sentencesEng”: [“I hate ironing,”, “I hate ironing, but it’s necessary”, “I hate ironing, but it’s necessary for work clothes.”]
},
{
“sentencesSpa”: [“Me gusta cocinar,”, “Me gusta cocinar, pero odio”, “Me gusta cocinar, pero odio tener que fregar”, “Me gusta cocinar, pero odio tener que fregar todos los platos sucios.”],
“sentencesEng”: [“I like to cook,”, “I like to cook, but I hate”, “I like to cook, but I hate having to wash”, “I like to cook, but I hate having to wash all the dirty dishes.”]
}
]
Of course. In this case you split the task into several subtasks. For example:
- identify the 10 headlines
- then save the content in between each headline into a file using the headline as name
- Zip all the files and provide a download link.
If you prompt it like this the initial reading of 1000 characters will become irrelevant as the model is looping over the requests specification. I did have some issues with the model identifying the headlines though and then some delimiter issues came on top. While this sample will likely work you may get better results with a better structure.
Hope this helps!
2 Likes