Looking for high-level guidance/pointers/examples how train custom query language

API
1

We use a custom query language at work. I have found a lot of examples/docs how to translate natural language to SQL, but I am looking for guidance guidance is how to teach the model how to do “SQL” (well not SQL but a custom query language)

Any guidance/pointers would be greatly appreciated, like: ANTLR definition would help?

Thanks in advance

1 Like
2

Have you looked at the OpenAI cookbook? https://cookbook.openai.com/
Also check out DeepLearning.AI short courses. Short Courses | Learn Generative AI from DeepLearning.AI

I use to know ANTLR but found that DCGs are so much better.

3

I looked at the OpenAI cookbook but did not find anything relevant, if there is something (even vaguely related) that I have potentially missed I would be grateful you can point it out.

4

First off, I have not modified an LLM for a specific purpose, I have created a ChatGPT plugin to use ChatGPT to help create Prolog which once one learns were the the mines in the minefield can be quite useful.

In the title you noted train so I took that to mean you understood embeddings so I took for granted that you would find what you needed.

For tutorials I would look at Question answering using embeddings-based search | OpenAI Cookbook for inspiration. I personally would also consider looking at RAG

If the information can be put into a PDF of tens of pages or less them consider using one of the many plugins that work with ChatGPT and PDFs.

Here is an example prompt for ChatGPT about ANTLR, this was not tested but from a quick glance someone with knowledge of ANTLR should be able to fix the mistakes.

Prompt

Create ANTLR 4 code in Java to read a CSV file.

Completion

ANTLR (ANother Tool for Language Recognition) is a powerful tool for creating parsers and interpreters for various languages and file formats. To use ANTLR 4 to read a CSV file in Java, you first need to define a grammar for the CSV format and then generate the corresponding Java code. Here’s a basic example of how to do this:

  1. Define a CSV grammar in ANTLR 4. Create a file named CSV.g4 with the following content:
grammar CSV;

file: header row* EOF;

header: row;

row: field (',' field)* '\n';

field: TEXT | STRING;

STRING: '"' (~["] | '""')* '"';

TEXT: ~[,\n"]+;

WS: [ \t\r]+ -> skip;

This grammar defines a simple CSV file format with a header row and multiple data rows. It recognizes fields as either plain text or strings enclosed in double quotes. It handles escaping of double quotes within strings.

  1. Generate Java code from the grammar using the ANTLR tool. You can do this using the ANTLR 4 command-line tool or by integrating it into your build system (e.g., Maven).

Assuming you have generated the parser and lexer using the command-line tool, you can create a Java program to read a CSV file as follows:

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.ParseTree;
import java.io.FileInputStream;
import java.io.IOException;

public class CSVReader {

    public static void main(String[] args) throws IOException {
        String inputFile = "data.csv"; // Replace with your CSV file path

        ANTLRInputStream input = new ANTLRInputStream(new FileInputStream(inputFile));
        CSVLexer lexer = new CSVLexer(input);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        CSVParser parser = new CSVParser(tokens);

        ParseTree tree = parser.file(); // Start with the "file" rule

        // Do something with the parse tree (e.g., print the CSV data)
        CSVVisitor visitor = new CSVVisitor();
        visitor.visit(tree);
    }
}
  1. Implement a custom visitor to do something with the parsed CSV data. Here’s a simple visitor example that prints the parsed data:
import org.antlr.v4.runtime.tree.AbstractParseTreeVisitor;

public class CSVVisitor extends CSVBaseVisitor<Void> {
    @Override
    public Void visitHeader(CSVParser.HeaderContext ctx) {
        System.out.println("Header: " + ctx.getText());
        return null;
    }

    @Override
    public Void visitRow(CSVParser.RowContext ctx) {
        System.out.println("Row: " + ctx.getText());
        return null;
    }

    @Override
    public Void visitField(CSVParser.FieldContext ctx) {
        System.out.println("Field: " + ctx.getText());
        return null;
    }
}

Make sure to replace "data.csv" in the CSVReader with the path to your CSV file.

  1. Compile and run the Java code, and it will parse the CSV file and print the header, rows, and fields.

Remember to download and include the ANTLR 4 runtime library in your project to make this code work.

Here is a link to the ChatGPT Prolog plugin but it was created in the early days of the ChatGPT plugins and I have not updated it since so not even sure if it still works.