Playground Assistant Inconsistent Responses

API
1

I have an assistant within Playground that I uploaded a .json file to the vector store to use for file search. I’ll attach the file below.

The assistant is giving inconsistent responses that do not match the file given to it. After grilling it, it will eventually spit out the correct information.

When I ask it questions about the data, such as:
“How many employees do I have?”
“What is the breakdown by user type?”
“How many employees were hired in July 2023?”
“How many employees were hired after October 2023?”

The correct answers should be:
67
Admin - 9, Drivers - 54, Technicians - 4
30
23

However, I am getting a range of answers including but not limited to:
" You have a total of 66 employees[1]."
" The breakdown of your employees by user type is as follows:

  • Drivers: 59
  • Technicians: 6
  • Administrators: 5

This totals to 66 employees[1]."
" In July 2023, you hired a total of 15 employees[1]."
" No employees were hired after October 2023[1]."

If I press it, it will eventually start to come up with relevant data:

User: “How many employees were hired after October 2023”

Assistant: " No employees were hired after October 2023[1]."

User: “Are you sure? I know there are some employees hired after that date”

Assistant: " You are correct. There were employees hired after October 2023. Specifically, the following employees were hired:

  1. Mark Webb - Driver, hired on November 14, 2023.
  2. Mike Byers - Driver, hired on October 16, 2023.
  3. Technician (Test Technician) - hired on October 23, 2023.

Thus, there are 3 employees hired after October 2023[1]."

User: “That’s still not correct. Can you search through again?”

Assistant: " After reviewing the data again, there are indeed employees hired after October 2023. The following employees were hired:

  1. Mark Webb - Driver, hired on November 14, 2023.
  2. Doug Stemple - Driver, hired on January 7, 2024.
  3. Mike Cameron - Administrator, hired on January 8, 2024.
  4. Dink Richardson - Administrator, hired on January 9, 2024.
  5. Don Oliver - Driver, hired on January 15, 2024.
  6. Alphonso Keene - Driver, hired on January 18, 2024.
  7. Bradley Swagerty - Driver, hired on February 6, 2024.
  8. Mike Newman - Driver, hired on March 7, 2024.
  9. Peter Cocco - Driver, hired on April 22, 2024.
  10. Yves Eugene - Driver, hired on April 23, 2024.
  11. Chris Shephard - Technician, hired on April 29, 2024.
  12. David Silva - Driver, hired on May 29, 2024.
  13. Chase Sears - Driver, hired on May 31, 2024.
  14. Toxie Driver - Driver, hired on June 11, 2024.
  15. Rodney Thompson - Driver, hired on June 26, 2024.
  16. Asnel Compere - Driver, hired on July 15, 2024.
  17. Eddie Pierre - Driver, hired on July 15, 2024.
  18. Matthew Johnson - Driver, hired on July 15, 2024.
  19. Jamond Ransom - Driver, hired on July 15, 2024.
  20. Princeston Green - Driver, hired on July 15, 2024.
  21. Angel Vivesrama - Driver, hired on July 15, 2024.

In total, 21 employees were hired after October 2023[1]."

Still not correct, but if I grill it, it’ll eventually come up with the correct data.

Here’s my current instructions:
"You are a helpful assistant. Be accurate, direct, and concise in your calculations. Don’t show any code. Just the results asked.

Only search these files included in the data files attached to this project. Your detailed factual answers must be answered only using information in the files I have provided and no other source.

Retrieve all data related to the query without limiting the number of results. Double check your findings and make sure you are counting correctly. Ensure a complete review of the data for accuracy and comprehensiveness."

Current model: gpt-4o-mini

I’m confused on if this is something I can fix. I have spent most of this morning adjusting the instructions to try to get it to do it right on the first pass and not have to be micromanaged verbally.

I’m trying to get an assistant up and going for data analysis, but how can I even begin to use it if I can’t trust it to be accurate with this seemingly simple data task?

TL / DR : The assistant is giving inconsistent responses that do not match the file given to it. After grilling it, it will eventually spit out the correct information.

JSON FILE BEING USED:
[
{
“ID”:1,
“USER_ID”:77,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“2/7/2023 8:39”,
“FIRSTNAME”:“Kyle “,
“LASTNAME”:“Frasier”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:null
},
{
“ID”:2,
“USER_ID”:93,
“COMPANY_ID”:53,
“EMAILADDRESS”:””,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“7/11/2023 9:21”,
“FIRSTNAME”:“Jeff”,
“LASTNAME”:“Tomkins”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:null
},
{
“ID”:3,
“USER_ID”:94,
“COMPANY_ID”:53,
“EMAILADDRESS”:“MIKE COONROD”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 11:38”,
“FIRSTNAME”:“MIKE”,
“LASTNAME”:“COONROD”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:145.0
},
{
“ID”:4,
“USER_ID”:97,
“COMPANY_ID”:53,
“EMAILADDRESS”:“CHARLIE BOYD”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:32”,
“FIRSTNAME”:“CHARLIE”,
“LASTNAME”:“BOYD”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:104.0
},
{
“ID”:5,
“USER_ID”:98,
“COMPANY_ID”:53,
“EMAILADDRESS”:“DALE CREWS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:36”,
“FIRSTNAME”:“DALE”,
“LASTNAME”:“CREWS”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:106.0
},
{
“ID”:6,
“USER_ID”:99,
“COMPANY_ID”:53,
“EMAILADDRESS”:“ROBERT LUCERO”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:42”,
“FIRSTNAME”:“ROBERT”,
“LASTNAME”:“LUCERO”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:117.0
},
{
“ID”:7,
“USER_ID”:100,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JOE CANNON”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:43”,
“FIRSTNAME”:“JOE”,
“LASTNAME”:“CANNON”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:105.0
},
{
“ID”:8,
“USER_ID”:101,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JOHN RHOADS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:45”,
“FIRSTNAME”:“JOHN”,
“LASTNAME”:“RHOADS”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:120.0
},
{
“ID”:9,
“USER_ID”:103,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JIM MIDEKE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:46”,
“FIRSTNAME”:“JIM”,
“LASTNAME”:“MIDEKE”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:118.0
},
{
“ID”:10,
“USER_ID”:104,
“COMPANY_ID”:53,
“EMAILADDRESS”:“BRUCE DEDMON”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:47”,
“FIRSTNAME”:“BRUCE”,
“LASTNAME”:“DEDMON”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:107.0
},
{
“ID”:11,
“USER_ID”:105,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JOHNNY SNELL”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:49”,
“FIRSTNAME”:“JOHNNY”,
“LASTNAME”:“SNELL”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:123.0
},
{
“ID”:12,
“USER_ID”:106,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JASON BAILEY”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:53”,
“FIRSTNAME”:“JASON”,
“LASTNAME”:“BAILEY”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:101.0
},
{
“ID”:13,
“USER_ID”:107,
“COMPANY_ID”:53,
“EMAILADDRESS”:“KENNY HUNTLEY”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:54”,
“FIRSTNAME”:“KENNY”,
“LASTNAME”:“HUNTLEY”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:129.0
},
{
“ID”:14,
“USER_ID”:108,
“COMPANY_ID”:53,
“EMAILADDRESS”:“ROBERT SOLORZANO”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:55”,
“FIRSTNAME”:“ROBERT”,
“LASTNAME”:“SOLORZANO”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:124.0
},
{
“ID”:15,
“USER_ID”:109,
“COMPANY_ID”:53,
“EMAILADDRESS”:“GREG SARDINAS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 16:56”,
“FIRSTNAME”:“GREG”,
“LASTNAME”:“SARDINAS”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:136.0
},
{
“ID”:16,
“USER_ID”:110,
“COMPANY_ID”:53,
“EMAILADDRESS”:“GARY PORTER”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:03”,
“FIRSTNAME”:“GARY”,
“LASTNAME”:“PORTER”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:149.0
},
{
“ID”:17,
“USER_ID”:111,
“COMPANY_ID”:53,
“EMAILADDRESS”:“SHAWN WARREN”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:05”,
“FIRSTNAME”:“SHAWN”,
“LASTNAME”:“WARREN”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:126.0
},
{
“ID”:18,
“USER_ID”:112,
“COMPANY_ID”:53,
“EMAILADDRESS”:“OWEN KUEKEN”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:06”,
“FIRSTNAME”:“OWEN”,
“LASTNAME”:“KUEKEN”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:113.0
},
{
“ID”:19,
“USER_ID”:113,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JAMES BENNETT”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:07”,
“FIRSTNAME”:“JAMES”,
“LASTNAME”:“BENNETT”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:103.0
},
{
“ID”:20,
“USER_ID”:114,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“7/11/2023 17:08”,
“FIRSTNAME”:“TIM”,
“LASTNAME”:“WILKINSON”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:127.0
},
{
“ID”:21,
“USER_ID”:116,
“COMPANY_ID”:53,
“EMAILADDRESS”:“GREGG HEINRICH”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:10”,
“FIRSTNAME”:“GREGG”,
“LASTNAME”:“HEINRICH”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:109.0
},
{
“ID”:22,
“USER_ID”:117,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“7/11/2023 17:11”,
“FIRSTNAME”:“M”,
“LASTNAME”:“Schwartz”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:122.0
},
{
“ID”:23,
“USER_ID”:123,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JOHNNIE LEWIS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:20”,
“FIRSTNAME”:“JOHNNIE”,
“LASTNAME”:“LEWIS”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:116.0
},
{
“ID”:24,
“USER_ID”:124,
“COMPANY_ID”:53,
“EMAILADDRESS”:“NICK KLEPAR”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:21”,
“FIRSTNAME”:“NICK”,
“LASTNAME”:“KLEPAR”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:112.0
},
{
“ID”:25,
“USER_ID”:127,
“COMPANY_ID”:53,
“EMAILADDRESS”:“KYLE FRASIER”,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“7/11/2023 17:25”,
“FIRSTNAME”:“KYLE”,
“LASTNAME”:“FRASIER”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:108.0
},
{
“ID”:26,
“USER_ID”:128,
“COMPANY_ID”:53,
“EMAILADDRESS”:“DAVID BAKER”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:26”,
“FIRSTNAME”:“DAVID”,
“LASTNAME”:“BAKER”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:102.0
},
{
“ID”:27,
“USER_ID”:131,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“7/11/2023 17:28”,
“FIRSTNAME”:“ANTHONY”,
“LASTNAME”:“ROBINSON”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:121.0
},
{
“ID”:28,
“USER_ID”:132,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JUSTIN KILGORE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/11/2023 17:29”,
“FIRSTNAME”:“JUSTIN”,
“LASTNAME”:“KILGORE”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:138.0
},
{
“ID”:29,
“USER_ID”:135,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JOSH BRIDGES”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/14/2023 12:56”,
“FIRSTNAME”:“JOSH”,
“LASTNAME”:“BRIDGES”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:158.0
},
{
“ID”:30,
“USER_ID”:136,
“COMPANY_ID”:53,
“EMAILADDRESS”:“TREVOR KEHL”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/14/2023 12:57”,
“FIRSTNAME”:“TREVOR”,
“LASTNAME”:“KEHL”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:157.0
},
{
“ID”:31,
“USER_ID”:137,
“COMPANY_ID”:53,
“EMAILADDRESS”:“WILLEX MURAT”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/14/2023 12:57”,
“FIRSTNAME”:“WILLEX”,
“LASTNAME”:“MURAT”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:159.0
},
{
“ID”:32,
“USER_ID”:184,
“COMPANY_ID”:53,
“EMAILADDRESS”:“TONY DUKES”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“9/26/2023 6:28”,
“FIRSTNAME”:“TONY”,
“LASTNAME”:“DUKES”,
“CELLNUMBER”:null,
“USERNAME”:“Tdukes”,
“EMPLOYEEID”:171.0
},
{
“ID”:33,
“USER_ID”:185,
“COMPANY_ID”:53,
“EMAILADDRESS”:“CURTISS DUPREE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“9/26/2023 6:30”,
“FIRSTNAME”:“CURTISS”,
“LASTNAME”:“DUPREE”,
“CELLNUMBER”:null,
“USERNAME”:“Cdupree”,
“EMPLOYEEID”:164.0
},
{
“ID”:34,
“USER_ID”:186,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JOSH EDWARDS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“9/26/2023 6:32”,
“FIRSTNAME”:“JOSH”,
“LASTNAME”:“EDWARDS”,
“CELLNUMBER”:null,
“USERNAME”:“Jedwards”,
“EMPLOYEEID”:163.0
},
{
“ID”:35,
“USER_ID”:187,
“COMPANY_ID”:53,
“EMAILADDRESS”:“WILL JOHNS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“9/26/2023 6:34”,
“FIRSTNAME”:“WILL”,
“LASTNAME”:“JOHNS”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:162.0
},
{
“ID”:36,
“USER_ID”:188,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JAN LOCKETT”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“9/26/2023 6:36”,
“FIRSTNAME”:“JAN”,
“LASTNAME”:“LOCKETT”,
“CELLNUMBER”:null,
“USERNAME”:“Jlockett”,
“EMPLOYEEID”:167.0
},
{
“ID”:37,
“USER_ID”:192,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JASPER SMITH”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“9/26/2023 6:49”,
“FIRSTNAME”:“JASPER”,
“LASTNAME”:“SMITH”,
“CELLNUMBER”:null,
“USERNAME”:“Jsmith”,
“EMPLOYEEID”:168.0
},
{
“ID”:38,
“USER_ID”:193,
“COMPANY_ID”:53,
“EMAILADDRESS”:“BRANDON THOMAS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“9/26/2023 6:51”,
“FIRSTNAME”:“BRANDON”,
“LASTNAME”:“THOMAS”,
“CELLNUMBER”:null,
“USERNAME”:“Bthomas”,
“EMPLOYEEID”:160.0
},
{
“ID”:39,
“USER_ID”:194,
“COMPANY_ID”:53,
“EMAILADDRESS”:“CALEB KENT”,
“USERTYPE”:“Technician”,
“CREATEDDATE”:“10/2/2023 4:17”,
“FIRSTNAME”:“CALEB”,
“LASTNAME”:“KENT”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:901.0
},
{
“ID”:40,
“USER_ID”:195,
“COMPANY_ID”:53,
“EMAILADDRESS”:“MIKE BOOZE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“10/4/2023 5:47”,
“FIRSTNAME”:“MIKE”,
“LASTNAME”:“BOOZE”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:174.0
},
{
“ID”:41,
“USER_ID”:197,
“COMPANY_ID”:53,
“EMAILADDRESS”:“TRAVARUS HARRIS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“10/11/2023 5:02”,
“FIRSTNAME”:“TRAVARUS”,
“LASTNAME”:“HARRIS”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:175.0
},
{
“ID”:42,
“USER_ID”:198,
“COMPANY_ID”:53,
“EMAILADDRESS”:“MIKE BAKER”,
“USERTYPE”:“Technician”,
“CREATEDDATE”:“10/11/2023 5:11”,
“FIRSTNAME”:“MIKE”,
“LASTNAME”:“BAKER”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:900.0
},
{
“ID”:43,
“USER_ID”:199,
“COMPANY_ID”:53,
“EMAILADDRESS”:“MIKE BYERS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“10/16/2023 4:46”,
“FIRSTNAME”:“MIKE”,
“LASTNAME”:“BYERS”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:177.0
},
{
“ID”:44,
“USER_ID”:201,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Technician”,
“CREATEDDATE”:“10/23/2023 4:08”,
“FIRSTNAME”:“Test”,
“LASTNAME”:“Technician”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:null
},
{
“ID”:45,
“USER_ID”:206,
“COMPANY_ID”:53,
“EMAILADDRESS”:“MARK WEBB”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“11/14/2023 7:19”,
“FIRSTNAME”:“MARK”,
“LASTNAME”:“WEBB”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:180.0
},
{
“ID”:46,
“USER_ID”:214,
“COMPANY_ID”:53,
“EMAILADDRESS”:“DOUG STEMPLE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“1/7/2024 11:17”,
“FIRSTNAME”:“DOUG”,
“LASTNAME”:“STEMPLE”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:181.0
},
{
“ID”:47,
“USER_ID”:216,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“1/8/2024 14:19”,
“FIRSTNAME”:“MIKE”,
“LASTNAME”:“CAMERON”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:null
},
{
“ID”:48,
“USER_ID”:217,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“1/9/2024 6:34”,
“FIRSTNAME”:“dink”,
“LASTNAME”:“richardson”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:1.0
},
{
“ID”:49,
“USER_ID”:218,
“COMPANY_ID”:53,
“EMAILADDRESS”:“DON OLIVER”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“1/15/2024 15:22”,
“FIRSTNAME”:“DON”,
“LASTNAME”:“OLIVER”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:182.0
},
{
“ID”:50,
“USER_ID”:219,
“COMPANY_ID”:53,
“EMAILADDRESS”:“ALPHONSO KEENE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“1/18/2024 6:11”,
“FIRSTNAME”:“ALPHONSO”,
“LASTNAME”:“KEENE”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:176.0
},
{
“ID”:51,
“USER_ID”:221,
“COMPANY_ID”:53,
“EMAILADDRESS”:“Bradley Swagerty”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“2/6/2024 5:47”,
“FIRSTNAME”:“Bradley”,
“LASTNAME”:“Swagerty”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:904.0
},
{
“ID”:52,
“USER_ID”:229,
“COMPANY_ID”:53,
“EMAILADDRESS”:“MIKE NEWMAN”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“3/7/2024 5:26”,
“FIRSTNAME”:“MIKE”,
“LASTNAME”:“NEWMAN”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:185.0
},
{
“ID”:53,
“USER_ID”:231,
“COMPANY_ID”:53,
“EMAILADDRESS”:“peter cocco”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“4/22/2024 11:42”,
“FIRSTNAME”:“PETER “,
“LASTNAME”:“COCCO”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:188.0
},
{
“ID”:54,
“USER_ID”:232,
“COMPANY_ID”:53,
“EMAILADDRESS”:“YVES EUGENE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“4/23/2024 9:49”,
“FIRSTNAME”:“YVES”,
“LASTNAME”:“EUGENE”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:187.0
},
{
“ID”:55,
“USER_ID”:234,
“COMPANY_ID”:53,
“EMAILADDRESS”:“Chris Shephard”,
“USERTYPE”:“Technician”,
“CREATEDDATE”:“4/29/2024 9:57”,
“FIRSTNAME”:“CHRIS”,
“LASTNAME”:“SHEPHARD”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:903.0
},
{
“ID”:56,
“USER_ID”:238,
“COMPANY_ID”:53,
“EMAILADDRESS”:“ALLEN MANDLEY”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“5/6/2024 14:43”,
“FIRSTNAME”:“ALLEN”,
“LASTNAME”:“MANDLEY”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:189.0
},
{
“ID”:57,
“USER_ID”:242,
“COMPANY_ID”:53,
“EMAILADDRESS”:””,
“USERTYPE”:“Administrator”,
“CREATEDDATE”:“5/16/2024 6:10”,
“FIRSTNAME”:“Toxie”,
“LASTNAME”:“Givens”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:null
},
{
“ID”:58,
“USER_ID”:246,
“COMPANY_ID”:53,
“EMAILADDRESS”:“DAVID SILVA”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“5/29/2024 12:43”,
“FIRSTNAME”:“DAVID”,
“LASTNAME”:“SILVA”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:193.0
},
{
“ID”:59,
“USER_ID”:247,
“COMPANY_ID”:53,
“EMAILADDRESS”:“CHASE SEARS”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“5/31/2024 7:24”,
“FIRSTNAME”:“CHASE”,
“LASTNAME”:“SEARS”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:194.0
},
{
“ID”:60,
“USER_ID”:248,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“6/11/2024 10:50”,
“FIRSTNAME”:“Toxie”,
“LASTNAME”:“Driver”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:1111.0
},
{
“ID”:61,
“USER_ID”:249,
“COMPANY_ID”:53,
“EMAILADDRESS”:“”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“6/26/2024 7:15”,
“FIRSTNAME”:“Rodney”,
“LASTNAME”:“Thompson”,
“CELLNUMBER”:,
“USERNAME”:null,
“EMPLOYEEID”:199.0
},
{
“ID”:62,
“USER_ID”:252,
“COMPANY_ID”:53,
“EMAILADDRESS”:“ASNEL COMPERE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/15/2024 9:17”,
“FIRSTNAME”:“ASNEL”,
“LASTNAME”:“COMPERE”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:186.0
},
{
“ID”:63,
“USER_ID”:253,
“COMPANY_ID”:53,
“EMAILADDRESS”:“EDDIE PIERRE”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/15/2024 9:17”,
“FIRSTNAME”:“EDDIE”,
“LASTNAME”:“PIERRE”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:190.0
},
{
“ID”:64,
“USER_ID”:254,
“COMPANY_ID”:53,
“EMAILADDRESS”:“MATTHEW JOHNSON”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/15/2024 9:27”,
“FIRSTNAME”:“MATTHEW”,
“LASTNAME”:“JOHNSON”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:195.0
},
{
“ID”:65,
“USER_ID”:255,
“COMPANY_ID”:53,
“EMAILADDRESS”:“JAMOND RANSOM”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/15/2024 9:28”,
“FIRSTNAME”:“JAMOND”,
“LASTNAME”:“RANSOM”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:201.0
},
{
“ID”:66,
“USER_ID”:256,
“COMPANY_ID”:53,
“EMAILADDRESS”:“PRICESTON GREEN”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/15/2024 9:33”,
“FIRSTNAME”:“PRINCESTON”,
“LASTNAME”:“GREEN”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:202.0
},
{
“ID”:67,
“USER_ID”:257,
“COMPANY_ID”:53,
“EMAILADDRESS”:“ANGEL VIVESRAMA”,
“USERTYPE”:“Driver”,
“CREATEDDATE”:“7/15/2024 9:40”,
“FIRSTNAME”:“ANGEL”,
“LASTNAME”:“VIVESRAMA”,
“CELLNUMBER”:null,
“USERNAME”:null,
“EMPLOYEEID”:914.0
}
]

2

Hi,

Unfortunately, this kind of numeric counting task is not what LLM’s are good at. You could try a combination of LLM and tools to select specific functions to run traditional software on your dataset with, as a kind of friendly front end. However, asking the model to count numeric values like this will almost certainly fail more than it works.

You could give the model a set of function tools that it can use to count strings that match set values given by the user, but in general the data it treated as context and not byte for byte data.

3

So how is it good for data analysis if it can not do basic functions like this?

4

You can show it data and it can find useful insights in that data, but if you notice the examples of the form of a few categories and perhaps 12 monthly splits of those. That is within the abilities of the model to handle internally.

However, your large JSON dataset contains many elements and you are requesting accurate counting of elements from a language model. The model has the ability to deal accurately with small numbers, and it also has a sense of what big numbers are, but asking it to perform accuracy reliant computation on data that can be achived by traditional software will usually end in failure.

You can get decent results from datasets when you ask questions about patterns in the data, but specific counting tasks over 10 or so are not suitable for current gen LLM’s

5

As the others have said - this is hard. But I assume you get this from some API call? Have you consider function calling? Connect the API and let the function calling handle the requests. You can tell it get all employees based on a type or hiring date etc?

1 Like
6

Thanks for the comments and feedback. Yeah we can try to figure this out. I just see all the “hype” about data analysis and then it does this. I am an engineer and I don’t see how this can crunch my numbers like some people are praising about it. Finding patterns is not majority of data analysis. We will scrap this project because the way this is useless. My son says it like teaching a 5 year old. I don’t think people have to worry about AI taking over the world if it can not do basic math. Math makes the world go around.