MT-NLG - Are we ever getting access to the 530 B parameters trained model?

I’ve been trying out different language models recently and I saw MT-NLG I understand it’s a crazy model but can’t seem to find the code that works, wondering if anyone has tried it

If you have to ask, don’t even try it. Not judging, but those models require a massive amount of GB to run on a GPU, which most consumers don’t have. I’m talking about 50GB+.
Your average GPU has 6GB.

We’ll need to wait until OpenAI (or somebody else) integrates it and then use it via an API. This also saves on headaches since you don’t need to tune it yourself.

No. Nvidia never released the earlier Megatrons either, like Turing-NLG, note. Their interest is in solely building the tooling as part of selling GPUs to well-heeled customers, not in necessarily releasing models or creating SOTA ones. If you look at the benchmarks MT-NLG is pretty disappointing anyway (highly undertrained, even before Chinchilla or GPT-4), so it is almost certainly not worth the hassle.

1 Like