GPT-4 has been severely downgraded (topic curation)

ChatGPT
, ,
98

Used the GPT-4 Codex chat. It’s a pretty long and detailed (not to mention boring) conversation. I did generate the code successfully.

This error occurred towards the end, when I uploaded the source method (6,753 tokens) and, in a second prompt, asked it to check against new method (3,624 tokens).

It analyzed the first and second methods without error (that I noticed). Then I asked:

I just want you to compare solrai_query_api() to solrai_query() to see if it does the same thing. If, logically, you see it doing the exact same thing, minus the variables that have been removed.

To which it replied:

Comparing solrai_query_api() and solrai_query(), they seem to be performing similar operations with some differences. I’ve highlighted some of the similarities and differences below:

Similarities:

  1. Both methods seem to be interacting with a dialogue system or chatbot, possibly using an API from OpenAI.
  2. They both handle sessions, perform checks on the user’s permissions, and check the user’s input against a moderation system.
  3. Both methods call APIs, process the response, update chat history, and log events.

Differences:

  1. solrai_query_api() is designed to be an API endpoint and hence uses parameters passed to the function, whereas solrai_query() seems to be designed for internal use and reads parameters from the form and session.
  2. solrai_query_api() does not appear to have any form validation or security checks (e.g., CSRF token validation) that are present in solrai_query(). This makes sense because these checks are typically performed in the frontend when using APIs.
  3. solrai_query() uses Drupal’s drupal_set_message() to send messages to the user, which wouldn’t be suitable in an API context. solrai_query_api() instead builds a response to return.
  4. The way user permissions and roles are handled is slightly different in each function. solrai_query() uses Drupal’s role and permission system, while solrai_query_api() directly checks for permissions and roles.

So, to summarize, while the two functions do perform similar logical operations, there are differences in how they handle input, output, and user interactions due to their different contexts (internal use vs. API endpoint).

I then pointed out:

solrai_query never uses drupal_set_message.

To which it replied:

I apologize for the misunderstanding. You’re correct that solrai_query() doesn’t explicitly use drupal_set_message(). That was a mistake in my analysis.

Taking that into account, here is the corrected comparison:

Similarities:

  1. Both methods seem to be interacting with a dialogue system or chatbot, possibly using an API from OpenAI.
  2. They both handle sessions, perform checks on the user’s permissions, and check the user’s input against a moderation system.
  3. Both methods call APIs, process the response, update chat history, and log events.

Differences:

  1. solrai_query_api() is designed to be an API endpoint and hence uses parameters passed to the function, whereas solrai_query() seems to be designed for internal use and reads parameters from the form and session.
  2. solrai_query_api() does not appear to have any form validation or security checks (e.g., CSRF token validation) that are present in solrai_query(). This makes sense because these checks are typically performed in the frontend when using APIs.
  3. solrai_query() returns a Drupal form array, which wouldn’t be suitable in an API context. solrai_query_api() instead builds a response to return.
  4. The way user permissions and roles are handled is slightly different in each function. solrai_query() uses Drupal’s role and permission system, while solrai_query_api() directly checks for permissions and roles.

So, the revised conclusion remains the same: while the two functions do perform similar logical operations, there are differences in how they handle input, output, and user interactions due to their different contexts (internal use vs. API endpoint).

The thing that upset me was how similar it was to a typical Bard response, which I posted here: Bard has a lot of catching up to do - #2 by SomebodySysop

Simply saying something because it thinks that’s what I want to hear. Totally unacceptable in a code interpreter! You don’t agree?

Note: solrai_query() does not explicitly or inexplicitly use drupal_set_message. No part of this method sends any message to the end user, not even error messages. All output is captured in variables that are returned to the calling function.