In this episode of the Information Bottleneck Podcast, we host Jack Morris, a PhD student at Cornell, to discuss adversarial examples (Jack created TextAttack, the first software package for LLM jailbreaking), the Platonic representation hypothesis, the implications of inversion techniques, and the role of compression in language models.
Links:
Jack's Website - https://jxmo.io/
TextAttack - https://arxiv.org/abs/2005.05909
How much do language models memorize? https://arxiv.org/abs/2505.24832
DeepSeek OCR - https://www.arxiv.org/abs/2510.18234
Chapters:
00:00 Introduction and AI News Highlights
04:53 The Importance of Fine-Tuning Models
10:01 Challenges in Open Source AI Models
14:34 The Future of Model Scaling and Sparsity
19:39 Exploring Model Routing and User Experience
24:34 Jack's Research: Text Attack and Adversarial Examples
29:33 The Platonic Representation Hypothesis
34:23 Implications of Inversion and Security in AI
39:20 The Role of Compression in Language Models
44:10 Future Directions in AI Research and Personalization
In this episode we talk with Randall Balestriero, an assistant professor at Brown University. We discuss the potential and challenges of Joint Embedding Predictive Architectures (JEPA). We explore the concept of JEPA, which aims to learn good data representations without reconstruction-based learning. We talk about the importance of understanding and compressing irrelevant details, the role of prediction tasks, and the challenges of preventing collapse.
In this episode, we talked with Michael Bronstein, a professor of AI at the University of Oxford and a scientific director at AITHYRA, about the fascinating world of geometric deep learning. We explored how understanding the geometric structures in data can enhance the efficiency and accuracy of AI models. Michael shared insights on the limitations of small neural networks and the ongoing debate about the role of scaling in AI. We also talked about the future in scientific discovery, and the potential impact on fields like drug design and mathematics
In this episode we host Tal Kachman, an assistant professor at Radboud University, to explore the fascinating intersection of artificial intelligence and natural sciences. Prof. Kachman's research focuses on multiagent interaction, complex systems, and reinforcement learning. We dive deep into how AI is revolutionizing materials discovery, chemical dynamics modeling, and experimental design through self-driving laboratories. Prof. Kachman shares insights on the challenges of integrating physics and chemistry with AI systems, the critical role of high-throughput experimentation in accelerating scientific discovery, and the transformative potential of generative models to unlock new materials and functionalities.
In this episode, we talked with Ahmad Birami, an ex-researcher at Google, to discuss various topics in AI. We explored the complexities of reinforcement learning, its applications in LLMs, and the evaluation challenges in AI research. We also discussed the dynamics of academic conferences and the broken review system. Finally, we discussed how to integrate theory and practice in AI research and why the community should prioritize a deeper understanding over surface-level improvements.
In this episode of the "Information Bottleneck" podcast, we hosted Aran Nayeb, an assistant professor at Carnegie Mellon University, to discuss the intersection of computational neuroscience and machine learning. We talked about the challenges and opportunities in understanding intelligence through the lens of both biological and artificial systems. We talked about topics such as the evolution of neural networks, the role of intrinsic motivation in AI, and the future of brain-machine interfaces.
We talked with Ariel Noyman, an urban scientist, working in the intersection of cities and technology. Ariel is a research scientist at the MIT Media Lab, exploring novel methods of urban modeling and simulation using AI. We discussed the potential of virtual environments to enhance urban design processes, the challenges associated with them, and the future of utilizing AI.
Links:
We discussed the inference optimization technique known as Speculative Decoding with a world class researcher, expert, and ex-coworker of the podcast hosts: Nadav Timor.
Papers and links:
In this episode, Ravid and Allen discuss the evolving landscape of AI coding. They explore the rise of AI-assisted development tools, the challenges faced in software engineering, and the potential future of AI in creative fields. The conversation highlights both the benefits and limitations of AI in coding, emphasizing the need for careful consideration of its impact on the industry and society.
Chapters
00:00Introduction to AI Coding and Recent Developments
03:10OpenAI's Paper on Hallucinations in LLMs
06:03Critique of OpenAI's Research Approach
08:50Copyright Issues in AI Training Data
12:00The Value of Data in AI Training
14:50Watermarking AI Generated Content
17:54The Future of AI Investment and Market Dynamics
20:49AI Coding and Its Impact on Software Development
31:36The Evolution of AI in Software Development
33:54Vibe Coding: The Future or a Fad?
38:24Navigating AI Tools: Personal Experiences and Challenges
41:53The Limitations of AI in Complex Coding Tasks
46:52Security Vulnerabilities in AI-Generated Code
50:28The Role of Human Intuition in AI-Assisted Coding
53:28The Impact of AI on Developer Productivity
56:53The Future of AI in Creative Fields
Allen and Ravid discuss the dynamics associated with the extreme need for GPUs that AI researchers utilize.
Allen and Ravid sit down and talk about Parameter Efficient Fine Tuning (PeFT) along with the latest updated in AI/ML news.
Allen and Ravid discuss a topic near and dear to their hearts, LLM Sampling!
In this episode of the Information Bottleneck Podcast, Ravid Shwartz-Ziv and Alan Rausch discuss the latest developments in AI, focusing on the controversial release of GPT-5 and its implications for users. They explore the future of large language models and the importance of sampling techniques in AI.
Chapters
00:00 Introduction to the Information Bottleneck Podcast
01:42 The GPT-5 Debacle: Expectations vs. Reality
05:48 Shifting Paradigms in AI Research
09:46 The Future of Large Language Models
12:56 OpenAI's New Model: A Mixed Bag
17:55 Corporate Dynamics in AI: Mergers and Acquisitions
21:39 The GPU Monopoly: Challenges and Opportunities
25:31 Deep Dive into Samplers in AI
35:38 Innovations in Sampling Techniques
42:31 Dynamic Sampling Methods and Their Implications
51:50 Learning Samplers: A New Frontier
59:51 Recent Papers and Their Impact on AI Research
