In this episode we talk about the paper "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean.
All content for Argmax is the property of Vahe Hagopian, Taka Hasegawa, Farrukh Rahman and is served directly from their servers
with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.
In this episode we talk about the paper "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean.
Todays paper: data2vec (https://arxiv.org/abs/2202.03555)Summary of the paperA multimodal SSL algorithm that predicts latent representation of different types of input.Highlights of discussionWhat are the motivations of SSL and multimodalHow does the student teacher learning work?What are similarities and differences between ViT, BYOL, and Reinforcement Learning algorithms.
Argmax
In this episode we talk about the paper "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, Jeff Dean.
