Is there any way to chat with my database datas by using chains like SqlDatabaseChain?

API
, , , ,
1

I tried to create a service to let users to chat with my database datas, but there is showing me an error: (node:72238) UnhandledPromiseRejectionWarning: Error: The combination of the database structure and your question is too big for the model text-davinci-003 which can compute only a max tokens of 4097. Im using the code below:

import { OpenAI } from "langchain/llms/openai";
import { SqlDatabase } from "langchain/sql_db";
import { SqlDatabaseChain } from "langchain/chains/sql_db";
import { DataSource } from "typeorm";

const langchainController = async () => {
  const datasource = new DataSource({
    type: "postgres",
    url: "my db url",
    logging: false,
    synchronize: true,
    extra: {
      ssl: {
        rejectUnauthorized: false,
      },
    },
  });

  const db = await SqlDatabase.fromDataSourceParams({
    appDataSource: datasource,
  });

  const chain = new SqlDatabaseChain({
    llm: new OpenAI({
      temperature: 0,
      openAIApiKey: "my api key",
    }),
    database: db,
  });

  const res = await chain.run("How many products are available now?");
  console.log(res);
};
export { langchainController };
2

The database library was perhaps designed for use with a chat model in mind, instead of a completion model, and uses a different tokenizer than davinci - or none at all.

Text is converted into a compressed form, where most english words can be represented as one token, if not just a few more. This is what the model is counting and what you have exceeded, the maximum context length of the AI model.

The context length must contain all of the input from the user conversation, must also contain the information retrieved from the database, and finally must have space reserved specifically for generating the output, a parameter max_tokens, and then that space can only be used for making output tokens, not accepting the input.

You’ll need to find a way of reducing the information presented to the AI model, or specify the max_tokens parameter (which is default 16 on these models, so something in the library may be setting it to more) to something that is just a bit more than the expected AI response and not something that takes a huge bite out of the context length and limits your possible input.

So it is possible, but you are asking specifically about the library you are using, which may have its own lines of support and methods to find more experienced users.