Creating an AI Assistant with OpenAI API: How to Upload Files for Knowledge Base?

API
, ,
2

You’ll primarily be working with the file management features of the OpenAI API to upload, and delete. You will use the vector Stores to create an searchable index of those files, and you will need to attach one or multiple vector stores to your assistant.

1. Uploading Files

  1. Prepare the File: Ensure your file is in a supported format and contains the data you want your assistant to use.
  2. Use the OpenAI API to Upload the File:
    import OpenAI from "openai";

const openai = new OpenAI({ apiKey: "your-api-key" });

async function uploadFile(filePath) {
    const fs = require('fs');
    const fileStream = fs.createReadStream(filePath);

    const response = await openai.files.create({
        file: fileStream,
        purpose: 'assistants'  // Specify 'assistants' to use this file for an AI assistant
    });

    console.log("Uploaded file ID:", response.data.id);
    return response.data;
}

uploadFile("/path/to/your/file.pdf");

2. Managing Files

Once files are uploaded, you can manage them by listing, updating, or deleting as needed to keep the assistant’s knowledge base relevant and up-to-date.

Listing Files:
To view or retrieve the files you’ve uploaded:

async function listFiles() {
    const response = await openai.files.list();
    console.log(response.data);
}

listFiles();

Updating Files:
Currently, OpenAI does not support directly updating an uploaded file. If you need to make changes, you should delete the existing file and upload a new version.

3. Using Files with the Assistant

To make these files useful, link them to your AI assistant via the tool_resources when you create or update your assistant.

Creating an Assistant with File Resources:

async function createAssistantWithFiles(vectorStoreId) {
    const response = await openai.assistants.create({
        model: "gpt-4-turbo",
        instructions: "You are a knowledgeable assistant that uses the provided files to answer questions.",
        tools: [{ type: "file_search" }],
        tool_resources: {
            file_search: {
                vector_store_ids: [vectorStoreId]  // Attach vector store containing your files
            }
        }
    });

    console.log("Assistant ID:", response.data.id);
}

createAssistantWithFiles("your-vector-store-id");

4. Vector Stores

Since files must be organized into vector stores to be used by an assistant, you need to manage these as part of your file handling.

Creating a Vector Store:

async function createVectorStore() {
    const response = await openai.beta.vectorStores.create({
        name: "My Knowledge Base",
        description: "A store of documents for the assistant to use."
    });

    console.log("Vector Store ID:", response.data.id);
    return response.data.id;
}

Adding Files to Vector Store:

async function addFilesToVectorStore(vectorStoreId, fileIds) {
    await openai.beta.vectorStores.fileBatches.createAndPoll(vectorStoreId, {
        file_ids: fileIds
    });

    console.log("Files added to Vector Store:", vectorStoreId);
}

Remember to handle the lifecycle of these files properly, ensuring they are updated or deleted as your application needs them to avoid much storage charges if you expect a lot of data being uploaded.

Happy building!

7 Likes