NVIDIA has announced the DGX B200, the successor to the DGX B200.
It’s fairly impressive to me, especially for the speed of inference and power consumption. What are your impressions?
2 Likes
Big honkin’ machine with a bigger honkin’ price tag!
NVIDIA will be selling these faster than they can build them for any price they want to set.
That said, I think by January 2026 these will be the driving force behind some models which would be otherwise impossible to build and deploy.
They will also result in GPT-4-level performance models being offered for GPT-3.5-Turbo-level prices.
In all, it’s an exciting time for AI.
3 Likes
Also, the performance gains are on 4-bit training and inference, vs FP16 or 32. Quant models will be built that way…
1 Like
When discussing NVIDIA’s products, it’s vital to consider their supply constraints. NVIDIA mainly focuses on design, while outsourcing the actual manufacturing process, primarily to TSMC.
As for TSMC, it’s crucial to take into account geopolitical risks, even though they are actively expanding their manufacturing facilities worldwide.
Designing an outstanding architecture doesn’t necessarily ensure an adequate supply to meet the demand.
1 Like