Dojo Processing Unit for GPT training?

Community
1

Tesla recently unveiled the new D1 chip which is a new approach to neural net hardware. First impressions – this is a huge step forward for the field of AI.

Official presentation here: Tesla AI Day 2021 - YouTube

I wonder if this tech could be considered as a good candidate for scaling up the GPT training, as well as inference process.

2 Likes
3

You’d probably be interested in Cerebras Wafer Scale Engine as well. Mammoth chips are all the rage nowadays.

3 Likes
4

Yeah, I saw that monster of a chip :slight_smile: The architecture seemed a bit forced tho, not so elegant…

2 Likes