Also, your order-of-magnitude arguments don’t perfectly apply because most sequences of tokens aren’t textlike at all. The space of natural language strings is much smaller, and might be small enough to map 1:1 to embedding vectors
Also, your order-of-magnitude arguments don’t perfectly apply because most sequences of tokens aren’t textlike at all. The space of natural language strings is much smaller, and might be small enough to map 1:1 to embedding vectors