Introducing Pathways: A next-generation AI architecture

2 Likes

LOLL I also shared a related topic: Investigating Google Pathways - Switch Transformers (Argmax MoE) + Sharding - Maths behind it - #3 by danielhanchen :slight_smile:

But ye although Jeff Dean didn’t specify what will be inside Pathways, I just took some wild guesses - ie Switch Transformers + Sharding + maybe FNet + Distillation + maybe Lottery Ticket Hypothesis + etc