Pre-training LLMs: One Model To Rule Them All? with Talfan Evans, DeepMind

https://is1-ssl.mzstatic.com/image/thumb/Podcasts221/v4/b9/51/ff/b951ffaf-ad4c-95e8-00d3-0e30b4564bea/mza_13799867449658691411.jpg/600x600bb.jpg

Thinking Machines: AI & Philosophy

Daniel Reid Cahn

27 episodes

1 month ago

“Thinking Machines,” hosted by Daniel Reid Cahn, bridges the worlds of artificial intelligence and philosophy - aimed at technical audiences. Episodes explore how AI challenges our understanding of topics like consciousness, free will, and morality, featuring interviews with leading thinkers, AI leaders, founders, machine learning engineers, and philosophers. Daniel guides listeners through the complex landscape of artificial intelligence, questioning its impact on human knowledge, ethics, and the future. We talk through the big questions that are bubbling through the AI community, covering topics like "Can AI be Creative?" and "Is the Turing Test outdated?", introduce new concepts to our vocabulary like "human washing," and only occasionally agree with each other. Daniel is a machine learning engineer who misses his time as a philosopher at King's College London. Daniel is the cofounder and CEO of Slingshot AI, building the foundation model for psychology.

All content for Thinking Machines: AI & Philosophy is the property of Daniel Reid Cahn and is served directly from their servers with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.

Technology

Society & Culture,

Philosophy

Pre-training LLMs: One Model To Rule Them All? with Talfan Evans, DeepMind

Thinking Machines: AI & Philosophy

37 minutes

1 year ago

Pre-training LLMs: One Model To Rule Them All? with Talfan Evans, DeepMind

Talfan Evans is a research engineer at DeepMind, where he focuses on data curation and foundational research for pre-training LLMs and multimodal models like Gemini. I ask Talfan:

Will one model rule them all?
What does "high quality data" actually mean in the context of LLM training?
Is language model pre-training becoming commoditized?
Are companies like Google and OpenAI keeping their AI secrets to themselves?
Does the startup or open source community stand a chance next to the giants?

Also check out Talfan's latest paper at DeepMind, Bad Students Make Good Teachers.