You could try piecing together theories by looking at open source transformer models, or your own models.
But since the latest GPT models are all private, who besides the OpenAI engineers know?
They aren’t transmitting the internal buffers and arrays in the API from these private models. It’s a black box.
But for fun, you could embed each token, create a vector library of all tokens, and then spin your own model based on this mapping of tokens to vectors.