This tutorial is a sequel to the original - Build your own AI assistant in 10 lines of code - Python:
In the previous tutorial we explored how to develop a simple chat assistant, accessible via the console, using the Chat Completions API.
In this sequel, we will solve the most asked question:
“How to conserve tokens and have a conversation beyond the context length of the Chat Completion Model?”
This post includes an easter egg for those who haven’t read the updated API reference.
Intro
Embeddings are a way of representing data as points in space where the locations of those points in space are semantically meaningful. For example, in natural language processing, words can be embedded as vectors(lists) of real numbers such that words that are semantically similar are located close together in the embedding space. This allows machine learning models to learn the relationships between words and to perform tasks such as text classification, sentiment analysis, and question answering.
In our case, embeddings provide a way to capture the meaning of text and enable us to find relevant messages based on their semantic similarity.
Prerequisites
To follow along with this tutorial, you’ll need the following:
- Python 3.7.1 or higher installed on your system
- The Python client library for the OpenAI API v0.27.0 (latest version at the time of writing)
- An OpenAI API key. If you don’t have one, sign up for the OpenAI API and get your API key.
Step 1: Set up the environment
Import the necessary libraries and set up the OpenAI API key. Make sure you have the openai and pandas libraries installed.
import openai
import json
from openai.embeddings_utils import distances_from_embeddings
import numpy as np
import csv
import pandas as pd
import os.path
import ast
openai.api_key = "YOUR_API_KEY" # Replace "YOUR_API_KEY" with your actual API key
Step 2: Create functions to store messages and lookup context
Now, we will define two functions: store_message_to_file() and find_context().
The store_message_to_file() function will take message object, obtain embeddings for its 'content', and store the message object and the embeddings obtained from "text-embedding-ada-002" model to a file as csv. We are using "text-embedding-ada-002" because of its economy and performance.
# save message and embeddings to file
def store_message_to_file(file_path, messageObj):
response = openai.Embedding.create(model="text-embedding-ada-002",
input=messageObj["content"])
emb_mess_pair: dict = {
"embedding": json.dumps(response['data'][0]['embedding']), # type: ignore
"message": json.dumps(messageObj)
}
header = emb_mess_pair.keys()
if os.path.isfile(file_path) and os.path.getsize(file_path) > 0:
# File is not empty, append data
with open(file_path, "a", newline="") as file:
writer = csv.DictWriter(file, fieldnames=header)
writer.writerow(emb_mess_pair)
else:
# File is empty, write headers and data
with open(file_path, "w", newline="") as file:
writer = csv.DictWriter(file, fieldnames=header)
writer.writeheader()
writer.writerow(emb_mess_pair)
The `find_context()` function will call the `store_message_to_file()` to store the user message along with its embedding. It will then take the embeddings from the file, calculate distances between the user's message embedding and previous message embeddings, and return a context message if context is found i.e. the messages which that are near enough.
In the following definition for find_context() I’m using the cityblock as the distance metric, but you can use other metrics based on your requirements.
# lookup context from file
def find_context(file_path, userMessageObj, option="both"):
messageArray = []
store_message_to_file(file_path, userMessageObj)
if os.path.isfile(file_path) and os.path.getsize(file_path) > 0:
with open(file_path, 'r') as file:
df = pd.read_csv(file_path)
df["embedding"] = df.embedding.apply(eval).apply(np.array) # type: ignore
query_embedding = df["embedding"].values[-1]
if option == "both":
messageListEmbeddings = df["embedding"].values[:-3]
elif option == "assistant":
messageListEmbeddings = df.loc[df["message"].apply(
lambda x: ast.literal_eval(x)['role'] == 'assistant'),
"embedding"].values[-1]
elif option == "user":
messageListEmbeddings = df.loc[df["message"].apply(
lambda x: ast.literal_eval(x)["role"] == 'user'),
"embedding"].values[:-2]
else:
return [] # Return an empty list if no context is found
distances = distances_from_embeddings(query_embedding,
messageListEmbeddings, # type: ignore
distance_metric="L1")
mask = (np.array(distances) < 21.6)[np.argsort(distances)]
messageArray = df["message"].iloc[np.argsort(distances)][mask]
messageArray = [] if messageArray is None else messageArray[:4]
messageObjects = [json.loads(message) for message in messageArray]
contextValue = ""
for mess in messageObjects:
contextValue += f"{mess['name']}:{mess['content']}\n"
contextMessage = [{
"role":
"system",
"name":
systemName,
"content":
f"{assistantName}'s knowledge: {contextValue} + Previous messages\nOnly answer next message."
}]
return contextMessage if len(contextValue) != 0 else []
Step 3: Initialize the conversation
Now, let’s initialize the conversation by setting up some initial variables and messages.
dir_path = "/path/to/your/directory/" # Replace with the directory path where you want to store the file
file_path = dir_path + input("Enter the file to use: ")
username = "MasterChief"
assistantName = "Cortana"
systemName = "SWORD"
message = {
"role": "user",
"name": username,
"content": input(f"This is the beginning of your chat with {assistantName}.\n\nYou:")
}
conversation = [{
"role": "system",
"name": systemName,
"content": f"You are {assistantName}, a helpful assistant to {username}. Follow directives by {systemName}"
}]
In this step, you need to provide the directory path where you want to store the file. Set the dir_path variable accordingly. The file_path variable is constructed by concatenating the directory path with the user-provided filename. The username, assistantName, and systemName variables can be customized to your preference.
We create the initial user message and system message to set the context for the conversation.
Step 4: Generate responses using Chat Completion API
Now, we can generate responses using the models on Chat Completions API.
We store responses from the model using store_message_to_file().
To retrieve context for the user message, we use the find_context() function, which also writes the user message and its embedding to the file.
These functions ensure that the conversation history is maintained and used to generate relevant responses.
fullResponse = ""
while message["content"] != "###":
context = find_context(file_path, message, option="both")
last = conversation[-2:] if len(conversation) > 2 else []
conversation = conversation[0:1] + last + context
conversation.append(message)
print(f"{assistantName}: ")
for line in openai.ChatCompletion.create(model="gpt-3.5-turbo-0613",
messages=conversation,
max_tokens=300,
stream=True):
token = getattr(line.choices[0].delta, "content", "")
print(token, end="")
fullResponse += token
print()
message["content"] = input("You: ")
assistantResponse = {
"role": "assistant",
"name": assistantName,
"content": fullResponse
}
store_message_to_file(file_path, assistantResponse)
conversation.append(assistantResponse)
fullResponse = ""
That’s it! You have now built a chat assistant using the Chat Completion Model. You can run the code and start interacting with the assistant in the console
