yes you’re right, currently we only can fine-tune the base model like ada, babbage, curie, or davinci but not the latest model ones like text-*, GPT-3.5, or GPT-4.
If you do work in Chatbot or any QnA conversation you can combine embedding + text/chat(choose one) completion model. I think the nice simple tutorial to begin with by reading and doing some experiments like in this tutorials.
Then after you’re getting used with how this combined models work, you can read through this discussions to have an idea how embedding takes a role in chat completion model.