I’m new to working with LLMs and AI so training by tokens is new to me. I’m not actually that interested in writing haikus, just using it as a way to learn the concepts. So the workaround is appreciated, but as a newcomer it seems like a gaping hole that GPT-4 is unable to understand syllables despite an otherwise robust understanding of the English language.
Although the AI understands words in terms of tokens, isn’t there a way the AI could be taught the syllables of words based on a dictionary that simply links an integer to each word?