The Cartesian Cafe is the podcast where an expert guest and Timothy Nguyen map out scientific and mathematical subjects in detail. This collaborative journey with other experts will have us writing down formulas, drawing pictures, and reasoning about them together on a whiteboard. If you’ve been longing for a deeper dive into the intricacies of scientific subjects, then this is the podcast for you. Topics covered include mathematics, physics, machine learning, artificial intelligence, and computer science.
Content also viewable on YouTube: www.youtube.com/timothynguyen and Spotify.
Timothy Nguyen is a mathematician and AI researcher working in industry.
Homepage: www.timothynguyen.com, Twitter: @IAmTimNguyen
Patreon: www.patreon.com/timothynguyen
All content for The Cartesian Cafe is the property of Timothy Nguyen and is served directly from their servers
with no modification, redirects, or rehosting. The podcast is not affiliated with or endorsed by Podjoint in any way.
The Cartesian Cafe is the podcast where an expert guest and Timothy Nguyen map out scientific and mathematical subjects in detail. This collaborative journey with other experts will have us writing down formulas, drawing pictures, and reasoning about them together on a whiteboard. If you’ve been longing for a deeper dive into the intricacies of scientific subjects, then this is the podcast for you. Topics covered include mathematics, physics, machine learning, artificial intelligence, and computer science.
Content also viewable on YouTube: www.youtube.com/timothynguyen and Spotify.
Timothy Nguyen is a mathematician and AI researcher working in industry.
Homepage: www.timothynguyen.com, Twitter: @IAmTimNguyen
Patreon: www.patreon.com/timothynguyen
Tai-Danae Bradley | Category Theory and Language Models
The Cartesian Cafe
2 hours 25 minutes 17 seconds
3 years ago
Tai-Danae Bradley | Category Theory and Language Models
Tai-Danae Bradley is a mathematician who received her Ph.D. in mathematics from the CUNY Graduate Center. She was formerly at Alphabet and is now at Sandbox AQ, a startup focused on combining machine learning and quantum physics. Tai-Danae is a visiting research professor of mathematics at The Master’s University and the executive director of the Math3ma Institute, where she hosts her popular blog on category theory. She is also a co-author of the textbook Topology: A Categorical Approach that presents basic topology from the modern perspective of category theory.
In this episode, we provide a compressed crash course in category theory. We provide definitions and plenty of basic examples for all the basic notions: objects, morphisms, categories, functors, natural transformations. We also discuss the first basic result in category theory which is the Yoneda Lemma. We conclude with a discussion of how Tai-Danae has used category-theoretic methods in her work on language modeling, in particular, in how the passing from syntax to semantics can be realized through category-theoretic notions.
Patreon: https://www.patreon.com/timothynguyen
Originally published on July 20, 2022 on YouTube: https://youtu.be/Gz8W1r90olc
Timestamps:
00:00:00 : Introduction
00:03:07 : How did you get into category theory?
00:06:29 : Outline of podcast
00:09:21 : Motivating category theory
00:11:35 : Analogy: Object Oriented Programming
00:12:32 : Definition of category
00:18:50 : Example: Category of sets
00:20:17 : Example: Matrix category
00:25:45 : Example: Preordered set (poset) is a category
00:33:43 : Example: Category of finite-dimensional vector spaces
00:37:46 : Forgetful functor
00:39:15 : Fruity example of forgetful functor: Forget race, gender, we're all part of humanity!
00:40:06 : Definition of functor
00:42:01 : Example: API change between programming languages is a functor
00:44:23 : Example: Groups, group homomorphisms are categories and functors
00:47:33 : Resume definition of functor
00:49:14 : Example: Functor between poset categories = order-preserving function
00:52:28 : Hom Functors. Things are getting meta (no not the tech company)
00:57:27 : Category theory is beautiful because of its rigidity
01:00:54 : Contravariant functor
01:03:23 : Definition: Presheaf
01:04:04 : Why are things meta? Arrows, arrows between arrows, categories of categories, ad infinitum.
01:07:38 : Probing a space with maps (prelude to Yoneda Lemma)
01:12:10 : Algebraic topology motivated category theory
01:15:44 : Definition: Natural transformation
01:19:21 : Example: Indexing category
01:21:54 : Example: Change of currency as natural transformation
01:25:35 : Isomorphism and natural isomorphism
01:27:34 : Notion of isomorphism in different categories
01:30:00 : Yoneda Lemma
01:33:46 : Example for Yoneda Lemma: Identity functor and evaluation natural transformation
01:42:33 : Analogy between Yoneda Lemma and linear algebra
01:46:06 : Corollary of Yoneda Lemma: Isomorphism of objects = Isomorphism of hom functors
01:50:40 : Yoneda embedding is fully faithful. Reasoning about this.
01:55:15 : Language Category
02:03:10 : Tai-Danae's paper: "An enriched category theory of language: from syntax to semantics"
Further Reading:
Tai-Danae's Blog: https://www.math3ma.com/categories
Tai-Danae Bradley. "What is applied category theory?" https://arxiv.org/pdf/1809.05923.pdf
Tai-Danae Bradley, John Terilla, Yiannis Vlassopoulos. "An enriched category theory of language: from syntax to semantics." https://arxiv.org/pdf/2106.07890.pdf
The Cartesian Cafe
The Cartesian Cafe is the podcast where an expert guest and Timothy Nguyen map out scientific and mathematical subjects in detail. This collaborative journey with other experts will have us writing down formulas, drawing pictures, and reasoning about them together on a whiteboard. If you’ve been longing for a deeper dive into the intricacies of scientific subjects, then this is the podcast for you. Topics covered include mathematics, physics, machine learning, artificial intelligence, and computer science.
Content also viewable on YouTube: www.youtube.com/timothynguyen and Spotify.
Timothy Nguyen is a mathematician and AI researcher working in industry.
Homepage: www.timothynguyen.com, Twitter: @IAmTimNguyen
Patreon: www.patreon.com/timothynguyen