EP8: Training Models at Scale | AWS for AI Podcast

https://is1-ssl.mzstatic.com/image/thumb/Podcasts211/v4/18/47/e4/1847e4ee-a9a0-c83f-c73d-a337383a90c8/mza_6562904297147485806.jpeg/600x600bb.jpg

AWS For AI

9 episodes

1 month ago

Decoding The Future of Artificial Intelligence with AWS:

Explore the frontiers of artificial intelligence with AWS For AI, your insider guide to the technologies reshaping our world.

Each episode brings you face-to-face with the brilliant minds behind groundbreaking AI innovations from pioneering researchers, to executives transforming businesses with generative AI.

Technology

Business

RSS

All content for AWS For AI is the property of AWS For AI and is served directly from their servers with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.

Decoding The Future of Artificial Intelligence with AWS:

Explore the frontiers of artificial intelligence with AWS For AI, your insider guide to the technologies reshaping our world.

Each episode brings you face-to-face with the brilliant minds behind groundbreaking AI innovations from pioneering researchers, to executives transforming businesses with generative AI.

Technology

Business

https://content.production.cdn.art19.com/images/9b/28/03/22/9b280322-1204-42e1-86f8-d68bb7154864/9e2da77f8baa13f185641ba414f769215fe069fad7ca7a50a0a173ecbca418c375e9f4c9496ab7b23fc0e96aa14a86d2c077d83e975c3763c3fefd1dd374f180.jpeg

EP8: Training Models at Scale | AWS for AI Podcast

AWS For AI

1 hour 4 minutes 15 seconds

2 months ago

EP8: Training Models at Scale | AWS for AI Podcast

Join us for an enlightening conversation with Anton Alexander, AWS's Senior Specialist for Worldwide Foundation Models, as we delve into the complexities of training and scaling large foundation models. Anton brings his unique expertise from working with the world's top model builders, along with his fascinating journey from Trinidad and Tobago to becoming a leading AI infrastructure expert. Discover practical insights on managing massive GPU clusters, optimizing distributed training, and handling the critical challenges of model development at scale. Learn about cutting-edge solutions in GPU failure detection, checkpointing strategies, and the evolution of inference workloads. Get an insider's perspective on emerging trends like GRPO, visual LLMs, and the future of AI model development. Don't miss this technical deep dive where we explore real-world solutions for building and deploying foundational AI models, featuring discussions on everything from low-level infrastructure optimization to high-level AI development strategies. Learn more: http://go.aws/47yubYq Amazon SageMaker HyperPod : https://aws.amazon.com/fr/sagemaker/ai/hyperpod/ The Llama 3 Herd of Models paper : https://arxiv.org/abs/2407.21783 Chapters: 00:00:00 : Introduction and Guest Background 00:01:18 : Anton Journey from Caribbean to AI 00:05:52 : Mathematics in AI 00:07:20 : Large Model Training Challenges 00:09:54 : GPU failures : Lama Herd of models 00:13:40 : Grey failures 00:15:05 : Model training trends 00:17:40 : Managing Mixture of Experts Models 00:21:50 : Estimate how many GPUs you need. 00:25:12 : Monitoring loss function 00:27:08 : Crashing trainings 00:28:10 : SageMaker Hyperpod story 00:32:15 : How we automate managing grey failures 00:37:28 : which metrics to optimize for 00:40:23 : Checkpointing Strategies 00:44:48 : USE Utilization, Saturation, Errors 00:50:11 : SageMaker Hyperpod for Inferencing 00:54:58 : Resiliency in Training vs Inferencing workloads 00:56:44 : NVIDIA NeMo Ecosystem and Agents 00:59:49 : Future Trends in AI 01:03:17 : Closing Thoughts