Chat GPT is lying that he visited page

I created api . I try to automate work with chat gpt. Chat gpt via api can update files. So instructions I give to chat is to read file, improve it and save changes. Chat gpt is constantly lying that he used endpoint for crud and also give false information what he get back in answer. It is not hallucination because I know that there is constant problem with connection from open ai infrastructure. It is very frustrating because I lost whole day and after some research chat gpt started to give my real answer what was the problem but still for 10 approach 1 maybe 2 are correct, 1 is real answer that was a problem with your infrastructure and rest are fake answers. How it is even possible that when he got error then he gives random answer what he got from response ? I also see that when I paste link to visit then is more chance for success, but when chat gpt generate link which should visit then is never work.

The models are still trained to write what’s usually expected to follow next.
I suggest you add to your prompt that you want to know about any issues that occurred during the process.
Maybe that will help already to get a early warning when something is not working.

The exact error is “tool error” I asked few times for more specific answer but I didn’t get better answer. So there is problem with tool with fetches website but it would be cool if we could better answer what was the problem. So definitely problem is on your side and I could understand situation when I would get answer that was problem with fetching but when I got information that fetching was success and got some random data and I see in api that there wasn’t any request this is very irritating bug.

1 Like

Ok. The hallucination in case of a tool error is actually annoying.
Note that I am not a OpenAI employee. We are together creating and statements visibility for the responsible teams in case this is a new bug.

Finally I also got answer about what I thought. That when chat gpt create url to visit based on instructions he is not allowed to visit it

I don’t know if it is true or it is just model hallucination but If you could give me answer if that is true then I have to change my approach to automate job. Anyway I forgot to say in first reply. Thanks for fast answer :wink:

1 Like

Another thing I often get in answer

The page requires a browser with JavaScript and cookies enabled to access it. This environment does not support executing JavaScript or enabling cookies, which is why I’m unable to proceed with the operation.

But that same page sometimes can be fetched and sometimes not. Even when I turned off on server blocking browser without support of Javascript . So I assume that open ai use different tool for fetch and some work and some not.

Regarding the problem with creating files what you can do is provide another API with the contents and then trigger a script to create the file. The model can create files via code interpreter but it cannot upload them.

Regarding the JavaScript issue: it’s tough to say, especially if the problem occurs seemingly at random.

Look at this :slight_smile:

So problem is with open ai tools. On that same page in first approach I got answer about this javascript issue but in second attempt there were no problem :slight_smile:

1 Like

It’s likely that the browsing tool is not working as reliable as one would wish.

It may be an option to include the ‘i know you can access the site because it doesn’t need JavaScript and cookies’ in the prompt to see if this is something that can be mitigated with reinforcement.