The Future of Life Institute (FLI) is a nonprofit working to reduce global catastrophic and existential risk from powerful technologies. In particular, FLI focuses on risks from artificial intelligence (AI), biotechnology, nuclear weapons and climate change. The Institute's work is made up of three main strands: grantmaking for risk reduction, educational outreach, and advocacy within the United Nations, US government and European Union institutions. FLI has become one of the world's leading voices on the governance of AI having created one of the earliest and most influential sets of governance principles: the Asilomar AI Principles.
All content for Future of Life Institute Podcast is the property of Future of Life Institute and is served directly from their servers
with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.
The Future of Life Institute (FLI) is a nonprofit working to reduce global catastrophic and existential risk from powerful technologies. In particular, FLI focuses on risks from artificial intelligence (AI), biotechnology, nuclear weapons and climate change. The Institute's work is made up of three main strands: grantmaking for risk reduction, educational outreach, and advocacy within the United Nations, US government and European Union institutions. FLI has become one of the world's leading voices on the governance of AI having created one of the earliest and most influential sets of governance principles: the Asilomar AI Principles.
What Happens After Superintelligence? (with Anders Sandberg)
Future of Life Institute Podcast
1 hour 44 minutes
4 months ago
What Happens After Superintelligence? (with Anders Sandberg)
Anders Sandberg joins me to discuss superintelligence and its profound implications for human psychology, markets, and governance. We talk about physical bottlenecks, tensions between the technosphere and the biosphere, and the long-term cultural and physical forces shaping civilization. We conclude with Sandberg explaining the difficulties of designing reliable AI systems amidst rapid change and coordination risks.
Learn more about Anders's work here: https://mimircenter.org/anders-sandberg
Timestamps:
00:00:00 Preview and intro
00:04:20 2030 superintelligence scenario
00:11:55 Status, post-scarcity, and reshaping human psychology
00:16:00 Physical limits: energy, datacenter, and waste-heat bottlenecks
00:23:48 Technosphere vs biosphere
00:28:42 Culture and physics as long-run drivers of civilization
00:40:38 How superintelligence could upend markets and governments
00:50:01 State inertia: why governments lag behind companies
00:59:06 Value lock-in, censorship, and model alignment
01:08:32 Emergent AI ecosystems and coordination-failure risks
01:19:34 Predictability vs reliability: designing safe systems
01:30:32 Crossing the reliability threshold
01:38:25 Personal reflections on accelerating change
Future of Life Institute Podcast
The Future of Life Institute (FLI) is a nonprofit working to reduce global catastrophic and existential risk from powerful technologies. In particular, FLI focuses on risks from artificial intelligence (AI), biotechnology, nuclear weapons and climate change. The Institute's work is made up of three main strands: grantmaking for risk reduction, educational outreach, and advocacy within the United Nations, US government and European Union institutions. FLI has become one of the world's leading voices on the governance of AI having created one of the earliest and most influential sets of governance principles: the Asilomar AI Principles.
