Answering customer service automatically


I want to create a service that reads periodically the open tickets in our customer service system, and answer it.

The closest post I find about it is: /t/fine-tuning-a-model-for-customer-service-for-our-specific-app/29376/15

I provide my current proof of concept below.

Some questions:

  1. Should I use the fine-tuning described here: OpenAI Platform - with a lot of example questions and answers
  2. Or should I use embeddings? - have now researched it yet, a bit unclear to me why, have to read up :nerd_face:
  3. How much information should be in the system prompt Vs. user?
  4. When I use the chat API, should i split up messages in the messages parameter, right now I concatenated messages of the current ticket + information about the customer’s order into one single user prompt
  5. What other improvements suggestions do you guys have? :slight_smile:
  6. Is it correct to use this API for this use case?

My idea is to provide all our customer service policies + order information + conversation (and maybe all tickets in the past few days from this customer for context), and then answer and tell the script which actions to do (cancelling order etc.)

import json
import os
from time import sleep

import openai
import requests

from shopify_utils import get_store

class OpenAI:
    def __init__(self):
        self.api_key = os.environ["OPENAI_API_KEY"]
        self.standard_system_prompt = """
        You are a customer service agent for clothing brand selling online. You are answering on email, 
        instagram direct and facebook messenger. 
        Format of response should be json:
         "action": <action>
        "message_to_customer": <message> or "" if no message
        "internal_note": <note> 
        Always include message to customer and an "internal_note" which explains the question and why you did answer as you did. 
        "close ticket"
        "cancel order"
        "block order"
        "no action"
        * If a customer has returned an item, we will refund the item minus 59 kr. But, if they have placed a new order
        after making the return, they get a full refund on the products they return. 
        self.examples = """
        * How to return? 

        Vi har skrevet utfyllende om retur her: ... 
        Der finner du skjemaet som skal fylles ut, og info om hvordan returen/byttet gjennomføres.
        Si ifra om det ikke skulle være tilstrekkelig informasjon på siden.
        * I have a complaint (my leggings is broken or similar)

        Beklager at du har hatt en så uheldig opplevelse med plagget.
        Vi har et reklamasjon-skjema som du kan fylle ut her: 
        Vi vil kontakte deg så fort vi har gått gjennom reklamasjonen. 
        * Wants to change order or wrote the wrong address
        block order and tell them to make a new order with the correct address.
        * If there is a long ongoing conversation with an agent
        just do nothing "no action". Then it is too complicated for AI to handle in some cases, unless 
        there is a clear thing to be said or the conversation can be closed because there is nothing more to do 
        * Customer asks if she an amount will be deducted from her refund if she places a new order after returning 
        Check order information above, if she has placed new order, tell her she gets a full refund for all items. 
        Typical spam:
        * Marketers on instagram: 
        Hello there! This is Eva, reaching out on behalf of @sultanslacks . We saw your profile and it’s so interesting...

        * comment on photos: usually just bots commenting on photos with emojis and "love your content" etc.
        𝑆𝐸𝑁𝐷 𝑖𝑡 𝑜𝑛 👉@oslo.fam_😍
        Dm it on
        * From noreply emails
        Just close the ticket, no message to customer.        

    def get_answer(self, message):
        if len(message) > 15000:
            message = message[:15000]
        openai.api_key = self.api_key
        response = openai.ChatCompletion.create(
                    "role": "system",
                    "content": self.standard_system_prompt
                    "role": "user",
                    "content": message
        return response.choices[0].message['content']

class Gorgias:
    def __init__(self):
        self.api_username = os.environ["GORGIAS_API_USERNAME"]
        self.api_token = os.environ["GORGIAS_API_TOKEN"]
        self.subdomain = os.environ["GORGIAS_SUBDOMAIN"]

    def get_tickets(self, view_id=1263608):
        tickets = []
        next_cursor = None
        while True:
            params = {"limit": 30,
                      "order_by": "created_datetime:desc",
                      "view_id": view_id
            if next_cursor:
                params['cursor'] = next_cursor
            response = requests.get(
                auth=(self.api_username, self.api_token),
            data = response.json()
            next_cursor = data.get('meta', {}).get('next_cursor')
            if not next_cursor:
        return tickets

    def get_messages(self, ticket_id):
        response = requests.get(
            auth=(self.api_username, self.api_token)
        messages_data = response.json()['data']

        if not messages_data:  # No messages, seems to be come in if there is a started, but not started insta conversation
            return None

        if messages_data[0]['subject'] is not None:
            message_text = "Subject: " + messages_data[0]['subject'] + "\n\n"
            message_text = ""

        if messages_data[0]['channel'] is not None:
            message_text += "Channel: " + messages_data[0]['channel'] + "\n\n"

        for message in messages_data:
            if message['from_agent']:
                sender = "Agent"
                sender = "Customer"
            if message['body_text'] is not None:
                message_text += sender + ": " + message['body_text'] + "\n\n"
        return message_text

    def post_reply(self, ticket_id, message):
        data = {
            "body_text": message,  # assuming the message is in text format
            "from_agent": True,
            "channel": "email",  # replace with the appropriate channel
            "source": {
                "via": "email",  # replace with the appropriate value
            "via": "email",  # replace with the appropriate value
            "sender": {
                "name": "Your Agent Name",  # replace with the appropriate value
                "email": ""  # replace with the appropriate value
            # if needed, include receiver details
            # "receiver": {
            #     "name": "Customer Name",  # replace with the customer name
            #     "email": ""  # replace with the customer email
            # }
        response =
            auth=(self.api_username, self.api_token),
        return response.json()

    def add_internal_note(self, ticket_id, note):
        data = {
            "body_text": note,
            "from_agent": True,
            "channel": "internal-note",
            "source": {
                "via": "chat",  # replace with appropriate value
            "via": "chat",  # replace with appropriate value
            "sender": {
                "name": "AI customer service",
                "email": ""  # has to be one of the agents
            "sent_datetime": "2023-06-18T00:00:00Z"  # replace with current date time in ISO 8601 format
        response =
            auth=(self.api_username, self.api_token),
        return response.json()

    def close_ticket(self, ticket_id):
        data = {"status": "closed"}
        response = requests.put(
            auth=(self.api_username, self.api_token),
        return response.json()

    def delete_ticket(self, ticket_id):
        response = requests.delete(
            auth=(self.api_username, self.api_token)
        return response.status_code  # returns HTTP status code. 204 for success.

    def cleanup_tickets(self, view_id=239088):
        tickets = self.get_tickets(view_id)
        for ticket in tickets:
            print(f"Deleting ticket: {ticket['id']}")  # optional, for logging
            sleep(0.2)  # optional, to avoid rate limiting

def auto_answer_open_tickets():
    all_open_view_id = 1263608
    tickets = gorgias.get_tickets(all_open_view_id)
    for ticket in tickets:
        latest_messages = gorgias.get_messages(ticket['id'])

        if "AI suggestion" in latest_messages:
            print("AI suggestion already sent, skipping.")

        if latest_messages is None:

        customer_email = ticket['customer']['email']
        order_info = ""
        if customer_email:
            orders = store.get_latest_orders(customer_email)
            if orders is not None:
                for order in orders:
                    order_info += f"\nOrder ID: {order['id']}\nOrder Name: {order['name']}\n"
                    order_info += f"Shipping Address: {order['billing_address']['address1']}, {order['billing_address']['city']}, {order['billing_address']['zip']}, {order['billing_address']['country']}\n"
                    order_info += f"Cancelled At: {order['cancelled_at']}\nCreated At: {order['created_at']}\nSubtotal Price: {order['current_subtotal_price']}\nFulfillment Status: {order['fulfillment_status']}\nPayment Gateway: {order['gateway']}\n"
                    for item in order['line_items']:
                        order_info += f"Item: {item['name']}, Quantity: {item['quantity']}\n"
                    if order.get('refunds'):
                        for refund in order['refunds']:
                            order_info += f"Refund Created At: {refund['created_at']}\n"
                            for refund_line_item in refund['refund_line_items']:
                                order_info += f"Refunded Item: {refund_line_item['line_item']['name']}, Quantity: {refund_line_item['quantity']}, amount: {refund_line_item['subtotal']} \n"

                print("No orders found for this email address.")

        # In the future, add more ticket metadata for better responses.

        ticket_information = f"""
        Customer email: {customer_email}
        Latest order(s) for customer, use the information here to answer customer: {order_info} 
        Latest Messages:\n{latest_messages}

        ai_reply = openAI.get_answer(ticket_information)
        ai_reply_dict = json.loads(ai_reply)

        # For now, we just put an internal note telling with the AI would do and continue,
        # If we see it does the correct thing, we can add the actual actions later.
            f"AI suggestion: {json.dumps(ai_reply_dict, indent=2, ensure_ascii=False)}"


        action = ai_reply_dict["action"]
        message_to_customer = ai_reply_dict["message_to_customer"]
        internal_note = ai_reply_dict["internal_note"]

        if internal_note:
            print(f"internal_note: {internal_note}")
            gorgias.add_internal_note(ticket['id'], internal_note)

        # Perform actions based on AI reply
        if action == "close ticket":
        elif action == "cancel order":
            # call your cancel_order function
        elif action == "block order":
            # call your block_order function

        # If there's a message to send, send it
        if message_to_customer:
            gorgias.post_reply(ticket['id'], message_to_customer)

if __name__ == "__main__":
    gorgias = Gorgias()
    openAI = OpenAI()
    store = get_store()

    print("Start answering tickets:")
    print("Start cleanup:")
    gorgias.cleanup_tickets(93801)  # spam view
    gorgias.cleanup_tickets(239088)  # old tickets view