4o/4o Mini know the dates

API
1

Hi everyone,

I’m new to the forum (sorry if this isn’t the correct category) but have been working witht the completions API for a good bit of time - not to say that I am proficient.

In comparing 4o mini to smaller, local Ollama models I realized that, whenever I asked 4o for a date in the past or future (e.g. what day was October 27th, 2007), it very accurately provided the day of the week.

I thought this was a bit odd because when I asked the same of other models, it was nonsense. Is there a special algoritm that OpenAI uses?

1 Like
2

It’s only fair to check this theory…

gpt-4o:

For Wednesday April 16, 2025, AI said Wednesday which is True!
For Saturday February 14, 1959, AI said Saturday which is True!
For Friday January 30, 2015, AI said Friday which is True!
For Sunday January 14, 1973, AI said Sunday which is True!
For Wednesday September 16, 2009, AI said Wednesday which is True!
For Saturday August 13, 2022, AI said Saturday which is True!
For Tuesday September 28, 1976, AI said Tuesday which is True!
For Thursday October 23, 1997, AI said Thursday which is True!
For Tuesday September 25, 2040, AI said Tuesday which is True!
For Monday December 19, 1960, AI said Monday which is True!
For Thursday August 11, 1994, AI said Thursday which is True!
For Saturday October 08, 2005, AI said Saturday which is True!
For Tuesday June 17, 2014, AI said Tuesday which is True!
For Monday January 08, 2001, AI said Monday which is True!
For Saturday January 19, 2013, AI said Saturday which is True!
For Monday May 02, 1994, AI said Monday which is True!
For Monday May 04, 1959, AI said Monday which is True!
For Saturday September 07, 1991, AI said Saturday which is True!
For Saturday November 27, 1954, AI said Saturday which is True!
For Monday June 09, 2031, AI said Monday which is True!
For Thursday April 17, 1997, AI said Thursday which is True!
For Tuesday December 11, 2012, AI said Tuesday which is True!
For Monday October 30, 1961, AI said Monday which is True!
For Tuesday January 07, 1992, AI said Tuesday which is True!
For Wednesday April 08, 2009, AI said Wednesday which is True!
For Wednesday August 10, 2022, AI said Wednesday which is True!
For Monday November 10, 1975, AI said Monday which is True!
For Monday April 29, 1985, AI said Monday which is True!
For Thursday April 08, 1976, AI said Thursday which is True!
For Saturday June 02, 2018, AI said Saturday which is True!

gpt-4o-mini:

For Wednesday November 26, 2031, AI said Thursday which is False!
For Saturday December 21, 2019, AI said Saturday which is True!
For Tuesday March 20, 2029, AI said Tuesday which is True!
For Monday April 12, 1982, AI said Monday which is True!
For Tuesday June 04, 1985, AI said Tuesday which is True!
For Tuesday March 27, 1945, AI said Tuesday which is True!
For Wednesday February 03, 1999, AI said Wednesday which is True!
For Monday April 09, 2007, AI said Monday which is True!
For Monday June 02, 2036, AI said Monday which is True!
For Thursday March 03, 2039, AI said Thursday which is True!
For Tuesday August 03, 2038, AI said Tuesday which is True!
For Thursday July 10, 1969, AI said Thursday which is True!
For Sunday December 24, 2000, AI said Sunday which is True!
For Monday April 30, 1945, AI said Monday which is True!
For Monday August 23, 1999, AI said Monday which is True!
For Wednesday March 31, 1954, AI said Wednesday which is True!
For Thursday October 27, 2005, AI said Thursday which is True!
For Friday April 20, 2040, AI said Saturday which is False!
For Friday October 18, 2030, AI said Friday which is True!
For Tuesday December 01, 1964, AI said Tuesday which is True!
For Tuesday June 06, 1967, AI said Tuesday which is True!
For Monday February 22, 1965, AI said Monday which is True!
For Friday February 15, 2036, AI said Saturday which is False!
For Friday August 12, 1955, AI said Friday which is True!
For Tuesday December 16, 1975, AI said Tuesday which is True!
For Sunday April 13, 2036, AI said Monday which is False!
For Tuesday January 18, 1994, AI said Tuesday which is True!
For Friday November 19, 1976, AI said Friday which is True!
For Friday September 16, 1983, AI said Friday which is True!
For Monday December 03, 1956, AI said Monday which is True!

I’d better sort:

For Wednesday April 17, 1901, AI said Wednesday which is True!
For Sunday May 08, 1904, AI said Sunday which is True!
For Thursday December 28, 1905, AI said Thursday which is True!
For Tuesday June 06, 1911, AI said Tuesday which is True!
For Wednesday January 28, 1920, AI said Wednesday which is True!
For Monday September 04, 1922, AI said Monday which is True!
For Tuesday September 07, 1937, AI said Tuesday which is True!
For Saturday December 03, 1938, AI said Saturday which is True!
For Thursday June 08, 1944, AI said Thursday which is True!
For Monday April 28, 1947, AI said Monday which is True!
For Thursday January 20, 1949, AI said Thursday which is True!
For Thursday October 26, 1950, AI said Thursday which is True!
For Sunday July 15, 1951, AI said Sunday which is True!
For Monday July 04, 1960, AI said Monday which is True!
For Thursday June 26, 1969, AI said Thursday which is True!
For Sunday August 03, 1969, AI said Sunday which is True!
For Thursday March 16, 1972, AI said Thursday which is True!
For Friday October 26, 1973, AI said Friday which is True!
For Saturday October 16, 1982, AI said Saturday which is True!
For Friday June 28, 1991, AI said Friday which is True!
For Tuesday August 12, 1997, AI said Tuesday which is True!
For Wednesday August 04, 1999, AI said Wednesday which is True!
For Saturday October 07, 2006, AI said Saturday which is True!
For Sunday May 03, 2015, AI said Sunday which is True!
For Friday July 17, 2020, AI said Friday which is True!
For Monday January 31, 2022, AI said Monday which is True!
For Tuesday October 04, 2022, AI said Tuesday which is True!
For Saturday December 07, 2041, AI said Sunday which is False!
For Tuesday August 11, 2048, AI said Monday which is False!
For Wednesday April 27, 2050, AI said Thursday which is False!

It is just too uncannily correct – until it is not. Does mini think with an algorithm with a year 2038 bug?

Structured output + single-purpose bot prompting.

import datetime, json
from random import randint as r
from openai import Client; client=Client()

def date_by_bot(date_string):
    schema = {
    "type": "object",
    "properties": {"day_of_the_week": {
            "type": "string", "description": "Output the day of the week",
            "enum": ["Sunday", "Monday", "Tuesday", "Wednesday", 
                     "Thursday", "Friday", "Saturday"]}
        }, "required": ["day_of_the_week"],"additionalProperties": False}
    strict_schema = {"name": "response", "strict": True, "schema": schema}
    response_format = {"type": "json_schema","json_schema": strict_schema}

    response = client.beta.chat.completions.parse(
      model="gpt-4o-mini", ###### "gpt-4o-mini" or "gpt-4o" only due to strict structured
      messages=[
        {"role": "system",
          "content": ("You are a megasavant in regards to calculations of times, "
              "dates and the days of the week throughout history, "
              "using advanced knowledge and instant recall to give correct answers."
              )
        },
        {"role": "user",
          "content": f"What is the day of the week for {date_string}?"
        }
      ],
      response_format=response_format, top_p=0.01, temperature=0.01, max_tokens=10)
    # USELESS FRIGGIN LIBRARY = response.choices[0].message.parsed.day_of_the_week
    bot_dow = json.loads(response.choices[0].message.content)['day_of_the_week']
    return bot_dow

def dow(year, month, day):
    return datetime.date(year, month, day).strftime("%A")

def pretty_date(year, month, day):
    """Returns a formatted date like 'October 20, 2022'."""
    return datetime.date(year, month, day).strftime("%B %d, %Y")

def random_date(year_range: list[int]) -> tuple[int, int, int]:
    """
    Generate a random date tuple, ints (YYYY,MM,DD) within a given year range.

    Args:    year_range (list[int]): A list of two integers representing a year range. 
            - Mixed two-digit and four-digit years are valid (e.g., [1980, 20]).
            - 30-99 -> 1930-1999, and 00-29 -> 2000-2029.
    """
    def normalize_year(y: int) -> int:
        return y + (1900 if 30 <= y <= 99 else 2000) if y < 100 else y
    y1, y2 = map(normalize_year, year_range);y1, y2 = sorted((y1, y2));m = r(1, 12)
    d = r(1, 28 if m == 2 else 31 if m in (1, 3, 5, 7, 8, 10, 12) else 30)
    return (r(y1, y2), m, d)

def check_bot(y,m,d):
    date = pretty_date(y,m,d); true_dow = dow(y,m,d)
    bot_dow = date_by_bot(date); bot_correct = true_dow == bot_dow
    print(f"For {true_dow} {date}, AI said {bot_dow} which is {bot_correct}!")
    return bot_correct

def run_test(num_of_tests=20):
    year_range = [1900, 2050]
    for result in sorted(random_date(year_range) for _ in range(num_of_tests)):
        check_bot(*result)

if __name__ == "__main__":
    run_test()
3

It was just so interesting to me that, for the limited testing I did, it was correct every single time without any external grounding! Maybe this was a big part of the training data (thinking with the date algorithm?) I doubt it would be as effective with dates farther in the past, especially before the switch to our current calendar.

1 Like