Is there a possibility that OpenAI will offer their models for on-premise usage, allowing users to fine-tune them on their own premises? It has been observed that GPT4 models have demonstrated superior performance compared to other open-source language models when it comes to code generation. Whether OpenAI will make GPT4 models available for download, enabling users to quickly conduct proof-of-concept for finetuning and not worrying about data privacy and legal implications.
2 Likes
Out of interest how much high performance GPU hardware do you have on premise?
I’ve seen estimates of needing over 3TB of GPU VRAM to run GPT4 unquantised but I’m not sure outsiders know for certain?
I believe the answer is no, btw, and you might want to investigate one of the larger open source models for this purpose. Maybe wait for Llama 3?
2 Likes
You are correct, we are a small company and don’t have high computing power, but was just curious, thanks for the reply and sorry for the delay. I have tried Llama 7B models, not satisfied with the performance.
This high GPU VRAM that merefield talks aboutisn’t to run a LLM (GPT), but to train it. Perhaps he’s thinking of enterprise level LLMs that run for customers. But if you want it for yourself 32-64 GB VRAM would be enough for a high end LLM atm.
There is already a huge open-source community that have packages that allow you to do this. Do you need some directions for this dhatraknilam?