GPT-4 Turbo Knowledge cutoff Issues

GPT-4-Turbo basically has all the quality of a random word generator at this point.

Above, the chance of a token is halved just by adding the word “assistant” within the system message. Probabilities at that token position:

January: 63.339285%
September: 36.114978%
(space): 0.515429%
early: 0.015157%
August: 0.006921%
December: 0.005903%
October: 0.000846%
April: 0.000467%
November: 0.000275%
March: 0.000259%
February: 0.000107%
June: 0.000091%
the: 0.000076%
July: 0.000066%
Jan: 0.000063%
May: 0.000021%
fall: 0.000010%
september: 0.000008%

It has clear tuning on two - and will use them in wild variation.

April and December are way down there…

1 Like