Bad behavior: answer `\n\n\n\n\n...`

Prompting
,
1

Hey everyone,

For the past six months, I’ve been working on a service that analyzes users’ trading history based on their custom questions. It’s been running smoothly, providing valuable insights to users.

This week, things got interesting! I received a new challenge: integrating this service with another custom-built AI service.

Here’s the gist:

  • The new AI service, like the analysis tool, is mine as well.
  • I’m using function tools to leverage GPT-4.
  • The goal is for GPT-4 to analyze user questions and:
    • Decide the most suitable function tool based on the question.
    • Choose the appropriate guidelines for answering the user (these guidelines vary depending on the chosen function).

I’m facing some challenges with this integration. For instance, I’m encountering situations where GPT-4 fails to call any function and returns an empty response (like the example below). This typically happens when the AI API is unable to make a function call.

{
  "ai_agent": {
    "user_question": "most successful tags with highest pnl on tuesdays between 9 and 10 am",
    "sql_command": null,
    "ai_answer": "\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n  \n\n\n\n\n\n\n\n\n\n\n\n\n\n \n\n\n\n\n\n\n\n\n\n\n\n\n\n",
    "is_redirect": false
  }
}
  • Has anyone else attempted a similar integration with GPT-4 and other AI services?
  • Any tips or tricks on effectively using function tools with GPT-4 for decision-making?
  • How can I improve error handling to ensure GPT-4 can consistently call the appropriate function and provide meaningful responses to user queries?

Looking forward to hearing your experiences and insights!

2

Welcome to the community!

Are you giving the model an “out?”, as in a reasonable option it can use when it can’t answer the query?

1 Like
3

Thanks.

Sure, I specify the kind of answer for the different options. Like:

If you didn't call any function, try as much as possible your answer could satisfy the user question using no more than 25 words. \ If you feel the user question is not clear enough, you can ask the user to provide more information. \

4

Here’s what I do. This doesn’t mean that others should do the same. This is just what works in my experience regarding interop

  1. Don’t use functions/tools or JSON mode.
    1. they make diagnosing issues difficult
    2. a proper prompt will yield the same results. If you can’t parse the result, you can at least inspect where it went wrong.
  2. Ensure that all fields are obvious. You can use comments in your schema to reinforce a field role. Just because something is obvious to you in your head, doesn’t mean it’s obvious to the model.
  3. Ensure that your prompt is answerable. Even if all fields are well specified, the model needs to understand what to do with them if they’re unfillable for some reason.
  4. Eliminate traps. In some instances, the model may fill a field in a certain way (runaway response) that will make it logically impossible to properly fill a subsequent field. You can avoid this with CoT.
  5. Use Chain of Thought where necessary: add additional fields as a staging ground where the model can “think” about a response, or aggregate the information, before filling the actually important fields. When parsing, you can just ignore these fields, or save them for audit/inspection.
  6. Understand and respect the models: Some models have specific quirks that force them to act in a certain way (e.g. the preacher’s disease). Try to work with their disabilities - you can often capture their behavior in fields and ignore it later, but you sometimes need to provide the models with an outlet, otherwise they may act out in unpredictable ways.

regarding your prompt:

I’m not super convinced this works well on its own. Does the model have any information that would motivate it to feel that the user question is not clear enough?

According to my understanding of LLMs, these models only “see” what’s relevant. Whatever context information you provide needs to be activated somehow in order to be incorporated into the response.

At this time, I’m not convinced that the models have enough metacognitive capacity to understand whether they are confused or not. As such, they have a hard time activating a “need more information” sentiment unless it’s obvious, such as seeing an incomplete checklist for example.

2 Likes
5

:8080

1m on 17:02:59, 05/03

Port :8080 opened on{…}.replit.dev:8080

Loaded existing assistant ID.

out of the blue this occurred whne i run my python scripts

  • Serving Flask app ‘main’

  • Debug mode: off

2024-05-03 16:03:02,488 - INFO - WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.

  • Running on all addresses (0.0.0.0)

  • Running on 127.0.0.1:8080

  • Running on 172.31.196.3:8080

2024-05-03 16:03:02,489 - INFO - Press CTRL+C to quit

2024-05-03 16:03:04,006 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:04] code 414, message Request-URI Too Long

2024-05-03 16:03:04,006 - INFO - 172.31.196.3 - - [03/May/2024 16:03:04] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:04,618 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:04] code 414, message Request-URI Too Long

2024-05-03 16:03:04,618 - INFO - 172.31.196.3 - - [03/May/2024 16:03:04] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:05,127 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:05] code 414, message Request-URI Too Long

2024-05-03 16:03:05,127 - INFO - 172.31.196.3 - - [03/May/2024 16:03:05] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:05,623 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:05] code 414, message Request-URI Too Long

2024-05-03 16:03:05,624 - INFO - 172.31.196.3 - - [03/May/2024 16:03:05] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:06,137 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:06] code 414, message Request-URI Too Long

2024-05-03 16:03:06,137 - INFO - 172.31.196.3 - - [03/May/2024 16:03:06] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:06,718 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:06] code 414, message Request-URI Too Long

2024-05-03 16:03:06,718 - INFO - 172.31.196.3 - - [03/May/2024 16:03:06] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:07,284 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:07] code 414, message Request-URI Too Long

2024-05-03 16:03:07,284 - INFO - 172.31.196.3 - - [03/May/2024 16:03:07] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:07,897 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:07] code 414, message Request-URI Too Long

2024-05-03 16:03:07,897 - INFO - 172.31.196.3 - - [03/May/2024 16:03:07] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:08,508 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:08] code 414, message Request-URI Too Long

2024-05-03 16:03:08,509 - INFO - 172.31.196.3 - - [03/May/2024 16:03:08] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:09,086 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:09] code 414, message Request-URI Too Long

2024-05-03 16:03:09,086 - INFO - 172.31.196.3 - - [03/May/2024 16:03:09] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:09,685 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:09] code 414, message Request-URI Too Long

2024-05-03 16:03:09,685 - INFO - 172.31.196.3 - - [03/May/2024 16:03:09] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:10,313 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:10] code 414, message Request-URI Too Long

2024-05-03 16:03:10,314 - INFO - 172.31.196.3 - - [03/May/2024 16:03:10] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:10,900 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:10] code 414, message Request-URI Too Long

2024-05-03 16:03:10,901 - INFO - 172.31.196.3 - - [03/May/2024 16:03:10] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:11,498 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:11] code 414, message Request-URI Too Long

2024-05-03 16:03:11,498 - INFO - 172.31.196.3 - - [03/May/2024 16:03:11] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:12,134 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:12] code 414, message Request-URI Too Long

2024-05-03 16:03:12,134 - INFO - 172.31.196.3 - - [03/May/2024 16:03:12] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:03:12,838 - ERROR - 172.31.196.3 - - [03/May/2024 16:03:12] code 414, message Request-URI Too Long

2024-05-03 16:03:12,838 - INFO - 172.31.196.3 - - [03/May/2024 16:03:12] “” HTTPStatus.REQUEST_URI_TOO_LONG -

2024-05-03 16:04:24,015 - ERROR - 172.31.196.3 - - [03/May/2024 16:04:24] code 414, message Request-URI Too Long 2024-05-03 16:04:24,015 - INFO - 172.31.196.3 - - [03/May/2024 16:04:24] “” HTTPStatus.REQUEST_URI_TOO_LONG - 2024-05-03 16:04:25,385 - ERROR - 172.31.196.3 - - [03/May/2024 16:04:25] code 414, message Request-URI Too Long 2024-05-03 16:04:25,388 - INFO - 172.31.196.3 - - [03/May/2024 16:04:25] “” HTTPStatus.REQUEST_URI_TOO_LONG - 2024-05-03 16:04:26,793 - ERROR - 172.31.196.3 - - [03/May/2024 16:04:26] code 414, message Request-URI Too Long 2024-05-03 16:04:26,794 - INFO - 172.31.196.3 - - [03/May/2024 16:04:26] “” HTTPStatus.REQUEST_URI_TOO_LONG - 2024-05-03 16:04:28,249 - ERROR - 172.31.196.3 - - [03/May/2024 16:04:28] code 414, message Request-URI Too Long 2024-05-03 16:04:28,249 - INFO - 172.31.196.3 - - [03/May/2024 16:04:28] “” HTTPStatus.REQUEST_URI_TOO_LONG - 2024-05-03 16:04:29,692 - ERROR - 172.31.196.3 - - [03/May/2024 16:04:29] code 414, message Request-URI Too Long 2024-05-03 16:04:29,692 - INFO - 172.31.196.3 - - [03/May/2024 16:04:29] “” HTTPStatus.REQUEST_URI_TOO_LONG -