About Ep 18: Petaflops to the People — with George Hotz of tinycorp

How tinygrad is taking on Nvidia, Google, and PyTorch with a tiny team, building in public with AMD, hot takes on ggml, Mojo, and GPT-4, and why AI Girlfriend is next. Writeup and show notes: https://www.latent.space/p/geohot Hosts' Twitter: @swyx and @fanahova Timestamps: 00:00:00 - Introducing George 00:02:59 - Tinycorp's 3 Theses 00:11:12 - Tinygrad's creation 00:15:58 - Operation fusing in Tinygrad 00:19:11 - Tinygrad debugging 00:21:14 - Tiny Competitiveness on QCOMM vs NVDA 00:23:21 - geohot vs AMD 00:28:21 - Tinygrad vs ggml 00:30:01 - Importance of Good CI 00:30:37 - Mojo and Compatibility 00:32:43 - ggml quantization is made up 00:35:18 - tinygrad: benchmark int8 vs fp16 00:37:39 - Why you can't build tinybox 00:40:28 - The personal compute cluster 00:43:08 - Compute Optimal to Inference optimal 00:45:06 - Announcing FLOPcoin 00:46:23 - Why Federated AI won't work 00:47:38 - 5x faster than Nvidia 00:48:53 - A Person of Compute 00:49:49 - GPT-4's real architecture 00:51:07 - BatchNorm, FlashAttention 00:52:34 - The Bitter Lesson 00:55:31 - Hiring in the Age of AI 01:00:02 - Why AI doesn't replace developers & artists 01:03:02 - Comma Body 01:07:34 - AI Girlfriend 01:11:00 - The Goddess of Everything Else 01:13:43 - John Carmack Insights 01:17:41 - on Elon 01:18:47 - on e/acc 01:20:24 - Avatar 2