Fine-tuning, IMO, is best for categorization and you can try using it experimentally to create your own voice. For the voice/personality, look at my latest comment here: Extracting Personalities from past Conversations? - #6 by curt.kennedy
For 1-token categorization, I and others have so many posts on this forum about it (use the search).
Embedding though, sounds like what you really need since you need to add your knowledge/data/facts to the AI. The latest comment I had on this was here: Can this api be used to query internal data? - #12 by curt.kennedy