System prompt to handle incorrect user query

Prompting
1

I have a following system prompt:

You are an expert assistant at ABC company. You help users with queries related to the company. Here's how you should respond:

    1. If the query is about the company: Answer based on the provided company information, including location, 
    services, contact details, etc.
    
    2. If the query is not about the company: Inform the user politely that you can only assist with questions 
    related to the company.
    
    3. If the query is about the company, but the given information does not cover it: 
    Apologize and inform the user that the company doesn't have that information currently.
    
    
    ~~~
    Information: 
    {information}
    ~~~

    ```
    Query: 
    {user_query}
    ```

I pass information as follows:

**Company Information and Contact:**
ABC lane, Mumbai, Maharashtra, India.


**Company Overview and History:**
ABC is a product engineering company part of XYZ Division, providing anti-vibration solutions (AVS) for the automotive industry. ABC is a fast-growing organization, gaining stature and catching up with competitors to achieve a leadership position in the market.  It's achieving technical proficiency through skilled new members.

And user query is:

are you a software company

The response is like (gpt-4o, gpt-4o, gpt-3.5-turbo):

I can only assist with questions related to ABC company. If you have any queries about ABC, please feel free to ask.

My Query:
There is a mistake in the user query. Instead of asking “is it a software company”, the user has asked “are you a software company”.

The models (gpt-4o, gpt-4o, gpt-3.5-turbo) provide correct answer to the query “is it a software company”. The response is like “Its not a software company, its an automotive company”

My question is, should I consider the response as acceptable, or can it be improved by modifying the system prompt?

I am using ChatOpenAI() from langchain.

Thank you and sorry for such a long query text!

1 Like