When using the API and a custom function, ChatGPT is rewriting my DALL-E prompt. Is there a way to stop that?

My bot collects some data from the user. Once the data I need is collected in the chat, it calls my custom function which then prompts DALL-E 3 to create an image using the data from the user.

The problem is that my DALL-E prompt is getting completely rewritten before the image is created. It adds a lot of flowery language (which is fine), but it often drops important parts of my prompt such as “do not include any humans in the image”.

Is there a way to stop this rewrite?