I wish this was the case. Or also, say you select what RLHF packages to include when running the model through the API. Similar to a fine-tune, but you can decide, from presets, and custom. Or completely disable RLHF.
But having the entire RLHF wall forced on you? Not good.
It’s fine if they want to bake this into the model the general public uses through ChatGPT. But don’t bake it into the API version that the devs use. Let us decide.