How to make the chat bot return relevant content?

API
1

Hi. I’m using
gpt-3.5-turbo-1106 for an e-store. It is supposed to suggest products to customers, which it does OK to some extent. However, we also have articles and recipes which we also would like to suggest. Now, with each API call I’m sending a JSON list with product names and URLs. I can’t send along recipes and articles because the list is huge. Also, 100% of the links to articles and recipes it suggests point to non-existing URLs.

Is there a way to post the title-URL article/recipes lists once and make the bot always use them?

I tried with fine tune, the file was accepted and the process completed alright, but nothing changed in the actual chat, even with the exact user prompt, sent in the fine tune JSON.

Suggested products are not always relevant. We have them categorized and tagged, but this categorizations exceed the limitations and cannot be sent along with each API call, not to mention this is not efficient at all.

Is there a way to send a file with all the products data, so that the bot returns more relevant products?

Thanks!

1 Like
2

You’ll have to use some kind of RAG setup.

Assistants API can automate much of that for you, or you can use local tools with the Completions API to search your database or file system (however you want to set it up).

The latter will probably conserve more tokens as you have less control over what the Assistant is doing and it tends to gobble up more tokens.

2 Likes
3

You also don’t have to rely on the “RAG” semantic search which has to try to match the user input to content just by similarity to what the user wrote, but instead you can offer your categorized knowledge directly to the AI as functions for exploring it, the code you write then fulfilling the listing.

For example, here is a first tier of a function schema that might have a return of a list of articles, a subset of the total. A second similar function could request from a list of actual documents by an index number you provide in the first to return the full texts.

This keeps the context length low and focused. Below was cranked out by an AI by my description of the nesting, and is a structured schema when used with the latest gpt-4o-2024-08-06 and set strict:true.

Here’s how the schema can be structured:

{
  "name": "knowledge_retrieval",
  "strict": false,
  "schema": {
    "type": "object",
    "properties": {
      "knowledge_retrieval": {
        "type": "object",
        "properties": {
          "type": {
            "type": "string",
            "enum": ["products", "help_articles", "recipes"],
            "additionalProperties": false
          },
          "categories": {
            "type": "string",
            "enum": [
              "Electronics", "Books", "Clothing", "Home Appliances",
              "Tech Support", "Product Guides", "Account Help", "Payment Issues",
              "Vegetarian", "Vegan", "Gluten-Free", "Quick Meals"
            ],
            "additionalProperties": false
          }
        },
        "required": ["type", "categories"],
        "additionalProperties": false
      }
    },
    "required": ["knowledge_retrieval"],
    "additionalProperties": false
  }
}

(I simplified to non-strict, because the playground validator is refusing despite tons of “additionalProperties”: false, a previous edit will show the allOf version…)

This is for the “tools=” parameter of a chat completions API call.

Although the usage will be self explanatory to the AI, when to use it is not, so description fields should be placed in the function, that the function is the first step of finding company documentation - that must be the primary source of answers.

This is an approach to defining a function’s output for knowledge retrieval, with specific categories based on the type of information being retrieved.

The return would fulfilled by your code, and the enum values tuned to what categories you actually have content classified into.

2 Likes
4

Thanks _j, I was thinking in the same direction. I have a database with words that relate to products/recipes/articles. When such a word is found in the user’s input, instead of sending the full product list, I can narrow the product choices on my end, which will provide a lot of headroom to add few articles and recipes in the system message. It’s still somewhat inefficient but will do the job for a start. Then I’ll definitely try the functions! Thanks a lot!

1 Like