👋🏼 Hey AI heads 🎙️
Join us for the very first Tech Beats Live 🔴, hosted by Kosseila—aka @CloudDude from @CloudThrill.
🎯 This chill & laid-back livestream will unpack LLM quantization 🔥:
We’ll be joined by two incredible guest stars to talk Enterprise vs Consumer Quantz 🗣️:
🔷 Eldar Kurtić – bringing the enterprise perspective with vLLM.
🔷 Colin Kealty – aka Bartowski, creator of the top-downloaded GGUF quantized LLMs on Hugging Face.
🫵🏼 Come learn and have some fun 😎.
𝐂𝐡𝐚𝐩𝐭𝐞𝐫𝐬:
(00:00) Host Introduction
(04:07) Eldar Intro
(07:33) Bartowski Intro
(13:04) What’s Quantization!
(16:19) Why LLM Quantization Matters?
(20:39) Training vs Inference – “The New Deal”
(27:46) Biggest Misconception About Quantization
(33:22) Enterprise Quantization in Production (vLLM)
(48:48) Consumer LLMs & Quantization (Ollama, llama.cpp, GGUF) – “LLMs for the People”
(01:06:45) BitNet 1-Bit Quantization from Microsoft
(01:28:14) How Long It Takes to Quantize a Model (Llama-3 70B) – GGUF or lm-compressor
(01:34:23) What Is I-Matrix & Why People Confuse It with IQ Quantization?
(01:39:36) What’s LoRA & LoRA-Q?
(01:42:36) What Is Sparsity?
(01:47:42) What Is Distillation?
(01:52:34) Extreme Quantization (Unsloth) of Big Models (DeepSeek) at 2-bits 70 % Size Cut
(01:57:27) Will Future Models (Llama-5) Be Trained on FP4 Tensor Cores?
(02:02:15) The Future of LLMs on Edge Devices (Google AI Edge)
(02:08:00) How to Evaluate the Quality of a Quantized Model
(02:26:09) Hugging Face’s Role in the World of LLM/Quantization
(02:33:46) Hugging Face’s Role in the World of LLM/Quantization
(02:36:41) LocalLlama Sub-Reddit Down (Moderator Goes Bananas)
(02:40:11) Guests’ Hope for the Future of LLMs & AI in General
📖 Check out the quantization blog: https://bitly/LLMQuant
#AI #LLM #Quantization #TechBeatsLive #LocalLlama #vLLM #Ollama
