Is it possible to develop a fully customized chatbot that leverages my company’s proprietary data without the need for fine-tuning, using the OpenAI API? I am looking to build a chatbot that can efficiently process and respond to queries based on internal company documents while ensuring accuracy, scalability, and real-time interaction. What are the best approaches to achieve this?
4 Likes
yes - multiple ways… RAG is what you want to search for.
fifty - fifty chance
It depends on your skills. But yes. It is possible. If you are a programmer, than I would say it takes about 3-4 hours of dedication to make an MVP
2 Likes
Can be faster also…
import os
import sqlite3
import openai
from flask import Flask, request, render_template
app = Flask(__name__)
openai.api_key = os.getenv("OPENAI_API_KEY")
# ---- Example system prompts ----
SYSTEM_PROMPT_TABLE_SELECTION = """
You are a helper in a Retrieval-Augmented Generation (RAG) system.
Your purpose is to select exactly ONE table name from:
- products
- faq
- blackhole
Rules:
1. If the user question is about clothes or boots, return "products".
2. If the user question is a general question not about products, return "faq".
3. If the user's message is not a question or is irrelevant, return "blackhole".
4. Return ONE word ONLY, no explanation.
"""
SYSTEM_PROMPT_PARAM_EXTRACTION = """
You are helping to construct a SQL WHERE clause for the 'products' table.
We have columns: name, size, color, price.
Given the user's query, extract the key product attributes they are interested in.
Output a JSON object with possible keys: name, size, color. Example:
{"name": "T-Shirt", "size": "M", "color": "Black"}
If the user only specifies color, then just return {"color": "Black"}.
If the user only specifies size, then just return {"size": "M"}.
Do not add extra commentary - return valid JSON only.
"""
def select_table_for_message(user_message, max_retries=3):
"""
Attempts to get exactly one valid table name from the model.
If the model fails to comply, we append a scolding user message and retry.
Returns the valid table name or None if ultimately unsuccessful.
"""
valid_tables = {"products", "faq", "blackhole"}
conversation = [
{"role": "system", "content": SYSTEM_PROMPT_TABLE_SELECTION},
{"role": "user", "content": user_message}
]
for _ in range(max_retries):
try:
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=conversation,
temperature=0.0,
)
content = response.choices[0].message.content.strip().lower()
if content in valid_tables:
return content
else:
# Scold the model and try again
scold = (
"§$%§$%$ WTH YOU PIECE OF §%§$% - I said ONE WORD!!! "
"Choose 'products', 'faq' or 'blackhole' only!"
)
conversation.append({"role": "user", "content": scold})
except Exception as e:
print(f"Error selecting table: {e}")
return None
return None
def extract_sql_params(user_message, max_retries=2):
"""
Uses a second LLM call to parse user_message and extract a JSON
object that defines relevant columns for a WHERE clause in `products`.
Returns a Python dict, e.g. {"size": "M", "color": "Black"}.
If extraction fails, returns empty dict.
"""
conversation = [
{"role": "system", "content": SYSTEM_PROMPT_PARAM_EXTRACTION},
{"role": "user", "content": user_message}
]
for _ in range(max_retries):
try:
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=conversation,
temperature=0.0,
)
content = response.choices[0].message.content.strip()
# Attempt to parse JSON
import json
parsed = json.loads(content)
if isinstance(parsed, dict):
return parsed
except Exception as e:
print(f"Error extracting SQL params: {e}")
# Scold or prompt again
scold = (
"Invalid JSON. Return a JSON object with possible keys: name, size, color."
)
conversation.append({"role": "user", "content": scold})
return {}
def query_database(table, where_params=None):
"""
Connects to 'demo.db' (or memory DB) and runs a simple query.
If table == 'products' and where_params are provided, build a WHERE clause.
If table == 'faq', maybe return top 3 results or search (example).
If table == 'blackhole', do nothing.
Returns a list of rows, each row is a tuple or dictionary as you prefer.
"""
db_path = "demo.db" # or adapt to your environment
conn = sqlite3.connect(db_path)
conn.row_factory = sqlite3.Row
cur = conn.cursor()
if table == "blackhole":
return []
elif table == "products":
base_query = "SELECT name, size, color, price FROM products"
conditions = []
values = []
# For each possible key, add to the WHERE clause
if where_params:
for key, val in where_params.items():
# e.g. "color = ?" and values.append(val)
conditions.append(f"{key} = ?")
values.append(val)
if conditions:
query = base_query + " WHERE " + " AND ".join(conditions)
else:
query = base_query
cur.execute(query, values)
rows = cur.fetchall()
return rows
elif table == "faq":
# For simplicity, just return everything.
# You could do keyword matching etc.
query = "SELECT question, answer FROM faq LIMIT 3"
cur.execute(query)
rows = cur.fetchall()
return rows
conn.close()
def combine_data_with_answer(table, rows, user_message):
"""
Optionally combine the retrieved rows with user's message and
ask the model to generate a final answer.
If table is blackhole, or no rows, you might just return a direct message.
"""
if table == "blackhole":
return "No relevant database query needed."
# Turn DB rows into a text snippet
if not rows:
snippet = "No results found in the database."
else:
snippet = f"Database results: {rows}"
# Could produce a final GPT-based answer incorporating snippet
conversation = [
{
"role": "system",
"content": "You are a helpful assistant that uses retrieved data to answer the user."
},
{
"role": "user",
"content": f"User question: {user_message}\n\nRetrieved data:\n{snippet}\n\nWrite a helpful, concise answer."
}
]
try:
response = openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=conversation,
temperature=0.7,
)
final_answer = response.choices[0].message.content.strip()
except Exception as e:
print(f"Error generating final answer: {e}")
final_answer = "Sorry, I couldn't process that."
return final_answer
@app.route("/", methods=["GET"])
def index():
return render_template("index.html")
@app.route("/ask", methods=["POST"])
def ask():
user_message = request.form.get("user_message", "")
if not user_message.strip():
return render_template("index.html", final_answer="No question provided.")
# 1) Determine the table
table = select_table_for_message(user_message)
if table is None:
# Could ask the user to clarify, but for this demo, just show an error
return render_template("index.html", final_answer="Could not determine table. Please try again.")
# 2) If 'products', extract params from user message
sql_results = []
if table == "products":
where_params = extract_sql_params(user_message)
sql_results = query_database(table, where_params)
elif table == "faq":
sql_results = query_database(table, {})
else:
# blackhole -> no query
pass
# 3) Generate a final answer with the retrieved data
final_answer = combine_data_with_answer(table, sql_results, user_message)
# 4) Render results to the user
return render_template(
"index.html",
table=table,
sql_results=[dict(row) for row in sql_results],
final_answer=final_answer,
)
if __name__ == "__main__":
app.run(debug=True, host="0.0.0.0", port=5000)
* disclaimer: a faq search on a relational db might not be the best example and the code was generated by o1… it won’t work - uses old models and is just meant as an example
2 Likes
I think the easiest way is to create an OpenAI Assistant using the filesearch fonction. You just upload your company’s propietary data and give the right instructions to the Assistant.
It is easy, fast and no code. Hope this helps.
1 Like
he asked for
So any other ideas?
I want to have that on my phone with a customizable avatar that has memory. One that learns from me and is super personalized. My digital twin sister. If you make it, put it on my phone!!
Pretty please.
@jochenschultz Regarding accuracy, our application is similar to what he is looking for. We use the OpenAI Assistants “out of the box” along with file_search and very carefully crafted instructions and uploads. No fine-tuning. Here’s how I rate our outcomes:
accuracy: 99%. We have seen very rare occasions where difficult questions have resulted in minor hallucinations. This particularly happens when the content has information all around the question but not speaking to it directly.
scalability: 100%. We have no limits except those that are imposed by the assistants api itself – 10k files, file sizes, token, and rpm limits.
real-time: This one depends on what you mean. If you take this to simply mean how fast it responds, our answers typically start streaming within 3-6 seconds and finish within 8-20s depending on length. If you mean, can it deal with changes to the content in real-time, the answer is yes – although this depends on a carefully crafted system that detects and uploads changes as they occur.
1 Like