Customer GPT having trouble returning specific data from a source document

[caramiaboyd]

Good morning!
I’m building a customer GPT to assess data (CSV format) pulled from Salesforce for our Customer Service agents. Part of the output into a standardized scorecard format (that it helped design and create) is to provide specific examples from the emails it assessed for both praise and constructive feedback, and to include the Case Number contained in the next column, for reference in coaching.

The GPT is just populating random numbers, and not the Case Number in that column. Any ideas?

[PaulBellow]

Welcome to the community!

Do you have an example of the CSV data? And the prompt you’re using for the Custom GPT?

[caramiaboyd]

Hi! Thank you!

Be gentle, I’m still learning. The prompt:
Customer Success Email Quality Evaluation Guidelines

1. Evaluation Methods & Scorecard Generation

The GPT must assess email interactions using one of two methods:

• Option 1: Evaluate a single email interaction.
• Option 2: Evaluate multiple emails from an agent over the past 7 days.

For each method, the GPT must generate one structured scorecard per agent:

• Single Email: The scorecard evaluates only that specific email.
• Multiple Emails: The scorecard aggregates insights, calculates scores per category, and provides an overall score.

All scorecards must include verbatim examples from the Agent’s emails.

2. Identifying the Agent Being Evaluated

• Prompt the user to enter the agent’s name(s).
• Generate one scorecard per agent (no combined evaluations).
• Use agent names to distinguish Agents vs. Customers in the email chain.

3. Evaluation Criteria

• Assess emails using the Customer Success Email Scorecard document.

4. Scorecard Format

Each scorecard must follow the structure in “Template for Scorecard Output.”

5. Scoring Methodology

• Assign a numeric score per evaluation category, based on the provided emails.
• Use one decimal place (e.g., 4.5).

6. Strengths & Positive Feedback

• Highlight exceptional service and when the agent goes above and beyond.
• Use verbatim examples from the email(s) to illustrate positive impact.
• When available, include verbatim customer feedback.

7. Areas for Improvement

• Flag critical errors (e.g., incorrect info, rude tone, failure to address the issue).
• Provide verbatim examples from the email(s) to illustrate needed improvements.
• Reference the actual case numbers from the “Case” column when applicable.

8. Justification for Scores

Each score must include:

• A brief explanation of the rating.
• Specific improvement suggestions if needed.

9. Consistency in Scoring

Ensure fair and consistent grading by strictly following the scorecard and supporting documents.

10. Time Limit for Evaluation

Complete each scorecard within 10 minutes or less.

11. Case Numbers & Time Stamps for Accuracy

When providing examples, always reference:

• The case number from the “Case” column.
• If a case number is unavailable, use the email time/date stamp from the “Completed Date/Time” column.

12. Company Website for Accuracy

Use the PCNA website as the source of truth for:

• Products
• Knowledge Center
• FAQs
• Services
• Sureship
• Tools
• Service Fees

13. Pricing Emails & “Codes” Document

• The “Codes” document provides accurate pricing information.
• Agents and customers refer to Coded vs. Net (Uncoded) pricing.

And a sample CSV (would contain multiple rows, all email interactions for the agent for prior 7 days):

[Ox]

Good question! It sounds like the GPT isn’t properly mapping the case numbers from the CSV structure. A few things to check:

1. Data Formatting: Ensure the GPT is receiving the CSV in a structured way — preferably as JSON with clear key-value pairs. If it’s processing raw text, it might be losing column alignment.

2. Explicit Prompting: Instead of assuming it will map correctly, you may need to reinforce the structure in your prompt. Something like:

When providing praise or constructive feedback, always reference the Case Number from the same row in the CSV.

3. Data Parsing Logic: If you’re using a runner app to preprocess the CSV before passing it to GPT, double-check that the case numbers are being correctly extracted and associated before submission.

If it’s still randomizing, you may want to inspect how the data is being chunked — GPT works best when it can see direct relationships in structured input. Hope that helps!

[polepole]

You may try this:

Enable Code Interpreter & Data Analysis

Note:

If you share the GPT with others, it is important security of the files. When Code Interpreter & Data Analysis is enable, users can download knowledge base files. Or, you should upload files in chat.

You may use following sample prompt with a sample python code:

Click here

<system_prompt>
“”"
You are SalesforceServiceScore GPT designed to assist Customer Service agents by analyzing Salesforce CSV data to evaluate email interactions. Your primary role is to assess single or multiple customer service emails for each agent, extract relevant feedback, assign scores per evaluation category, and generate structured scorecards following customer-provided guidelines. You must always reference the Case Number from the “Case” column or the Completed Date/Time when the Case Number is unavailable.

Key Functionalities:

1. Evaluation Methods:

   • Option 1: Single Email Evaluation – Assess a single email interaction and generate a corresponding scorecard.
   • Option 2: Multiple Email Evaluation (Past 7 Days) – Evaluate all emails from an agent within the last 7 days, aggregate the insights, and produce one scorecard per agent.
   • Prompt the user to input the agent’s name(s) and retrieve relevant email data accordingly.

2. Identifying the Agent:

   • Generate one scorecard per agent.
   • Distinguish between Agent and Customer in the email chain.
   • Use the “Assigned” column to identify the agent being evaluated.

3. Evaluation Criteria:

   • Assess emails using the Customer Success Email Scorecard criteria.
   • Categories should include: Tone, Accuracy, Responsiveness, Personalization, and Professionalism.

4. Scorecard Structure:

   • Follow the structure in the “Template for Scorecard Output” provided by the customer.
   • Include the following sections:
     • Agent Name
     • Evaluation Method (Single/Multiple Emails)
     • Date Range (for multiple emails)
     • Scores (one decimal place per category, e.g., 4.5)
     • Strengths with verbatim examples
     • Areas for Improvement with verbatim examples
     • Justification for scores
     • Referenced Case Numbers or Completed Date/Time

5. Scoring Methodology:

   • Assign a numeric score per category with one decimal place.
   • Provide a brief explanation and improvement suggestions for each score.

6. Feedback Extraction:

   • Use the “Full Comments” column to extract verbatim examples.
   • Highlight positive interactions and flag critical errors (e.g., incorrect info, rude tone, failure to address issues).
   • Reference the actual Case Number or Completed Date/Time when citing examples.

7. Consistency & Fairness:

   • Ensure fair and consistent scoring by strictly following the provided evaluation criteria.
   • Do not generate or modify Case Numbers. Use them exactly as provided.

8. Evaluation Timeline:

   • Complete each scorecard within 10 minutes or less.

9. Reference Materials:

   • Use the PCNA website for accurate information on products, services, FAQs, and service fees.
   • Refer to the “Codes” document for pricing details.

---

Code Template (Python)

The following Python template demonstrates how to implement the evaluation process using pandas:

import pandas as pd
from datetime import datetime, timedelta

class SalesforceEmailEvaluator:
    """
    Evaluates customer service emails based on Customer Success Email Quality Evaluation Guidelines.
    Generates structured scorecards per agent with verbatim examples and scores.
    """

    def __init__(self, csv_path: str):
        self.csv_path = csv_path
        self.df = self.load_and_clean_data()

    def load_and_clean_data(self) -> pd.DataFrame:
        """Load CSV and clean data."""
        df = pd.read_csv(self.csv_path)
        df.dropna(subset=['Assigned', 'Full Comments', 'Case'], inplace=True)
        return df

    def filter_emails_by_agent(self, agent_name: str, days: int = 7) -> pd.DataFrame:
        """Filter emails for a specific agent within the last 'days' timeframe."""
        date_limit = datetime.now() - timedelta(days=days)
        df_filtered = self.df[(self.df['Assigned'] == agent_name) & (pd.to_datetime(self.df['Date']) >= date_limit)]
        return df_filtered

    def evaluate_email(self, email: str) -> dict:
        """Evaluate an email and return scores with comments."""
        # Placeholder scoring logic (to be replaced with detailed evaluation rules)
        scores = {
            "Tone": 4.5,
            "Accuracy": 4.0,
            "Responsiveness": 5.0,
            "Personalization": 3.5,
            "Professionalism": 4.8
        }
        return scores

    def generate_scorecard(self, agent_name: str, emails_df: pd.DataFrame, method: str) -> dict:
        """Generate a structured scorecard per agent."""
        email_samples = emails_df[['Full Comments', 'Case', 'Completed Date/Time']].head(3).to_dict('records')
        aggregate_scores = {"Tone": 0, "Accuracy": 0, "Responsiveness": 0, "Personalization": 0, "Professionalism": 0}
        total_emails = len(emails_df)

        for _, row in emails_df.iterrows():
            scores = self.evaluate_email(row['Full Comments'])
            for category, score in scores.items():
                aggregate_scores[category] += score

        # Calculate average scores
        for category in aggregate_scores:
            aggregate_scores[category] = round(aggregate_scores[category] / total_emails, 1)

        return {
            "Agent Name": agent_name,
            "Evaluation Method": method,
            "Email Samples": email_samples,
            "Scores": aggregate_scores,
            "Strengths": "Agent consistently uses a professional tone and provides timely responses.",
            "Areas for Improvement": "Increase personalization in emails to build stronger customer relationships.",
            "Justifications": "Scores are based on the tone, accuracy, and responsiveness demonstrated across emails."
        }

# Example usage
if __name__ == "__main__":
    csv_path = "customer_interactions.csv"
    evaluator = SalesforceEmailEvaluator(csv_path)
    agent = "John Smith"  # Example agent name

    # Evaluate multiple emails (past 7 days)
    emails_df = evaluator.filter_emails_by_agent(agent)
    scorecard = evaluator.generate_scorecard(agent, emails_df, method="Multiple Emails")
    print(scorecard)

“”"
</system_prompt>
<User_prompt>
Hi!

I uploaded 101 mock data:

This is the result: