If you’re using GPT-3.5, you can finetune the model in the API by providing enough examples formatted in JSON in a way that matches the example in the documentation. It’ll probably cost you a few dollars.
However, finetuning for GPT-4 models is currently limited to very few users (you get access after doing a lot of GPT-3.5 finetuning and applying through a form explaining your use-case).