180: Reinforcement Learning Programming Throwdown podcast

1:37:36

Intro topic: Getting an entry-level job News/Links: Mario Kart 64 Fully Decompiled https://gbatemp.net/threads/mario-kart-64-decompilation-project-reaches-100-completion.671104/ Q-Learning is not yet scalable https://seohong.me/blog/q-learning-is-not-yet-scalable/ Grover’s Algorithm https://www.youtube.com/watch?v=RQWpF2Gb-gU&vl=en OrangePi has a RISC-V SBC https://linuxgizmos.com/orangepi-rv2-a-cost-effective-risc-v-board-with-m-2-2280-slot-and-dual-gigabit-ethernet/ Book of the Show Patrick The Will of the Many (James Islington) https://amzn.to/44Dznsz Jason The Intelligence Trap https://amzn.to/3TqoKCB Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick Pokemon Odyssey https://www.reddit.com/r/PokemonROMhacks/comments/1l9zdta/pok%C3%A9mon_odyssey_final_release/ Jason Netflix Games https://play.google.com/store/apps/dev?id=6891422865930303475&hl=en_US Topic: Why Speed up development Catch errors faster than type checking/compiling Writing tedious boilerplate code Ask questions and learn local information Look good for hiring managers How Extensions for VSCode & other IDEs for inline suggestions Chat with a selection/file Command-line Tools run at the root directory Local vs Cloud Examples Copilot (VSCode extension) Use the experimental mode Cursor (Custom IDE) Jumps to suggest changes in other places Similar to copilot experimental mode RooCode (VSCode extension) ★ Support this podcast on Patreon ★…

1
181: Memory Management 1:46:21

8 weeks ago1:46:21

1:46:21

Intro topic: Video Game Prices News/Links: Step one: Jump in the Lava - Abyssoft https://youtu.be/WdadpHLAfdA?si=oXYnhB0EdkR_RaPE Scalable world models for continuous control https://www.tdmpc2.com/ Clever code is probably the worst code you could write - Engineer’s Codex https://read.engineerscodex.com/p/clever-code-is-probably-the-worst A new, open source text-to-speech model called Dia has arrived to challenge ElevenLabs, OpenAI and more https://venturebeat.com/ai/a-new-open-source-text-to-speech-model-called-dia-has-arrived-to-challenge-elevenlabs-openai-and-more/ Book of the Show Patrick The Muscle Ladder - Jeff Nippard https://amzn.to/44Dznsz Jason Metaphysics of War https://amzn.to/4jMjvZ5 Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick Pokemon Trading Card Game Pocket Jason Phi-4 https://huggingface.co/spaces/microsoft/phi-4-multimodal Topic: Memory Management Motivation Avoid thrashing / crashes Allocate resources efficiently Keep high uptime Where OS Level Heap management Virtual Memory Language/Compiler Level Cpp Garbage collection Ownership Tools Instrumentation Export to Datadog / Grafana Python: psutil & tracemalloc Valgrind What to do when your program uses too much memory? Reduce data sizes Compression References Lazy initializer Generators & Back Pressure Ring buffers Arena allocators Disk based caching ★ Support this podcast on Patreon ★…

1
180: Reinforcement Learning 1:52:22

16 weeks ago1:52:22

1:52:22

Intro topic: Grills News/Links: You can’t call yourself a senior until you’ve worked on a legacy project https://www.infobip.com/developers/blog/seniors-working-on-a-legacy-project Recraft might be the most powerful AI image platform I’ve ever used — here’s why https://www.tomsguide.com/ai/ai-image-video/recraft-might-be-the-most-powerful-ai-image-platform-ive-ever-used-heres-why NASA has a list of 10 rules for software development https://www.cs.otago.ac.nz/cosc345/resources/nasa-10-rules.htm AMD Radeon RX 9070 XT performance estimates leaked: 42% to 66% faster than Radeon RX 7900 GRE https://www.tomshardware.com/tech-industry/amd-estimates-of-radeon-rx-9070-xt-performance-leaked-42-percent-66-percent-faster-than-radeon-rx-7900-gre Book of the Show Patrick: The Player of Games (Ian M Banks) https://a.co/d/1ZpUhGl (non-affiliate) Jason: Basic Roleplaying Universal Game Engine https://amzn.to/3ES4p5i Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Pokemon Sword and Shield Jason: Features and Labels ( https://fal.ai ) Topic: Reinforcement Learning Three types of AI Supervised Learning Unsupervised Learning Reinforcement Learning Online vs Offline RL Optimization algorithms Value optimization SARSA Q-Learning Policy optimization Policy Gradients Actor-Critic Proximal Policy Optimization Value vs Policy Optimization Value optimization is more intuitive (Value loss) Policy optimization is less intuitive at first (policy gradients) Converting values to policies in deep learning is difficult Imitation Learning Supervised policy learning Often used to bootstrap reinforcement learning Policy Evaluation Propensity scoring versus model-based Challenges to training RL model Two optimization loops Collecting feedback vs updating the model Difficult optimization target Policy evaluation RLHF & GRPO ★ Support this podcast on Patreon ★…

1
179: Project Planning 1:43:00

22 weeks ago1:43:00

1:43:00

Intro topic: Lego event space & retail store: https://www.instagram.com/bambeecave News/Links: StackOverflow Question Count Going Down https://gist.github.com/hopeseekr/f522e380e35745bd5bdc3269a9f0b132 DeepSeek claims its ‘reasoning’ model beats OpenAI’s o1 on certain benchmarks https://techcrunch.com/2025/01/20/deepseek-claims-its-reasoning-model-beats-openais-o1-on-certain-benchmarks/ Computer Science Papers Every Developer Should Read https://newsletter.techworld-with-milan.com/p/computer-science-papers-every-developer Nvidia Cosmos - an AI platform to change the future of robots and cars - wins Best of CES 2025 https://www.zdnet.com/article/nvidia-signs-largest-car-maker-toyota-to-use-its-self-driving-tech/ Book of the Show Patrick: Alice’s Adventures in a differentiable wonderland https://www.sscardapane.it/alice-book/ Jason: A Beautiful Day in the Neighborhood (Hulu/Netflix/etc) Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Digseum https://store.steampowered.com/app/3361470/Digseum/ Jason: Sqlitedict - Python dictionaries saved to disk Topic: Project Planning and Management Why? Gathering feedback Identifying risks Deciding future headcount Documenting / discovering dependencies Critical path Schedule Reduce the bullwhip effect How it works SMART goals specific, measurable, achievable, relevant, and time-bound MT is most important Gantt Charts Scrum Agile Kanban Tools Whiteboard (the generic IRL one) Post it notes JIRA Asana OpenProject Dealing with uncertainty Buffering Issues with recursive padding Project planning Post-Mortems ★ Support this podcast on Patreon ★…

1
178: Working from Home 1:45:15

31 weeks ago1:45:15

1:45:15

Intro topic: Smart homes News/Links: SpaceX Starship Flight Test Five / Six https://www.youtube.com/watch?v=pIKI7y3DTXk ShareDB https://github.com/share/sharedb Orion AR Glasses https://about.fb.com/news/2024/09/introducing-orion-our-first-true-augmented-reality-glasses/ Blade and Sorcery 1.0 is out https://www.meta.com/experiences/blade-sorcery-nomad/2031826350263349/ Book of the Show Patrick: The Book that Wouldn’t Burn by Mark Lawrence https://amzn.to/4fry2XW Jason: Masters of Doom https://amzn.to/3YxuD3c Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Balatro https://www.playbalatro.com/ Jason: Cursor IDE https://www.cursor.com/ Topic: Working from Home Intro Background & WFH experiences Is it Panacea? Realizing it works better for some than others Internally Motivated Scheduling Communications Home Setup Dedicated space Handling Non-work Distractions Keyboards, Monitors, Music, … Desk related things The specter of RTO ★ Support this podcast on Patreon ★…

1
177: Vector Databases 1:28:26

35 weeks ago1:28:26

1:28:26

Intro topic: Buying a Car News/Links: Cognitive Load is what Matters https://github.com/zakirullin/cognitive-load Diffusion models are Real-Time Game Engines https://gamengen.github.io/ Your Company Needs Junior Devs https://softwaredoug.com/blog/2024/09/07/your-team-needs-juniors Seamless Streaming / Fish Speech / LLaMA Omni Seamless: https://huggingface.co/facebook/seamless-streaming Fish: https://github.com/fishaudio/fish-speech LLaMA Omni: https://github.com/ictnlp/LLaMA-Omni Book of the Show Patrick: Thought Emporium Youtube https://youtu.be/8X1_HEJk2Hw?si=T8EaHul-QMahyUvQ Jason: Novel Minds https://www.novelminds.ai/ Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Escape Simulator https://pinestudio.com/games/escape-simulator/ Jason: Cursor IDE https://www.cursor.com/ Topic: Vector Databases (~54 min) How computers represent data traditionally ASCII values RGB values How traditional compression works Huffman encoding (tree structure) Lossy example: Fourier Transform & store coefficients How embeddings are computed Pairwise (contrastive) methods Forward models (self-supervised) Similarity metrics Approximate Nearest Neighbors (ANN) Sub-Linear ANN Clustering Space Partitioning (e.g. K-D Trees) What a vector database does Perform nearest-neighbors with many different similarity metrics Store the vectors and the data structures to support sub-linear ANN Handle updates, deletes, rebalancing/reclustering, backups/restores Examples pgvector: a vector-database plugin for postgres Weaviate, Pinecone Milvus ★ Support this podcast on Patreon ★…

1
176: MLOps at SwampUp 1:58:37

41 weeks ago1:58:37

1:58:37

James Morse: Software Engineer at Cisco System Administrator to DevOps Difference between DevOps and MLOps Getting Started with DevOps Luke Marsden: CEO of Helix ML How to start a business at 15 years old BTRFS vs ZFS MLOps: the intersection of software, DevOps and AI Fine-tuning AI on the Cloud Some advice for folks interested in ML Ops Yuval Fernbach: CTO MLOps & JFrog Starting Qwak Going from a jupyter notebook to production ML Supply Chain Getting started in Machine Learning Stephen Chin: VP of DevRel at Neo4J Developer Relations: The Job What is a Large Language Model? Knowledge graphs and the Linkage Model How to Use Graph databases in Enterprise How to get into ML Ops ★ Support this podcast on Patreon ★…

1
175: Resume Writing 1:40:55

47 weeks ago1:40:55

1:40:55

175: Resume Writing Intro topic: DSLR Photography vs Camera Phone News/Links: Free Internet while flying by abusing edits to your profile name https://robertheaton.com/pyskywifi/ Making Animated Characters with AI Art https://www.youtube.com/watch?v=zSN76gb_Z28 On 10x Engineers https://stackoverflow.blog/2024/06/19/the-real-10x-developer-makes-their-whole-team-better/ The Beauty and Challenges of AI-Generated Artistic Gymnastics https://www.youtube.com/watch?v=YwJIYj3hPAU Book of the Show Patrick: The Three Body Problem by Cixin Liu https://amzn.to/3xNEoRB Jason: The Checklist Manifesto https://amzn.to/3W2JjpM Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Super Mario Bros. Wonder (Nintendo Switch) https://amzn.to/3S9VJLf Jason: Amazon Q https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.amazon-q-vscode Topic: Resume Writing (Courtesy of Matthew C.) Why have a resume? Many jobs require it to get into the consideration Today many are screened for keywords automatically Log for future you What is a resume? One-page description Key accomplishments & experiences Comparison to CV References How to write a good resume? Do’s Include your github if it has good contributions Be specific (dates, locations, skills) Isolate your specific contributions Be accurate/honest Be concise Be ready to discuss any point you have on the resume List hobbies/activities/extracurriculars Don’ts Have mistakes (especially dates) Use images (most companies use text extraction) Use it as a design portfolio Put social qualities (e.gs. hard-working, motivated, friendly) Use fancy templates/tools Resources Manager Tools: How to scan resumes https://www.manager-tools.com/2016/05/how-scan-resume-part-1 Google docs Latex/Lyx for CVs How to think about your career and how it impacts your future resume writing (career planning) Technologies and architectures more than specifics of project details How various choices may age over time ★ Support this podcast on Patreon ★…

1
174: Devops 1:25:47

1 year ago1:25:47

1:25:47

Intro topic: Social Media Auto Responder LLM News/Links: Amazon releases Amazon Q https://press.aboutamazon.com/2024/4/aws-announces-general-availability-of-amazon-q-the-most-capable-generative-ai-powered-assistant-for-accelerating-software-development-and-leveraging-companies-internal-data Cheap RiscV “Super Cluster” from Bitluni DIY 256-Core RISC-V super computer https://www.youtube.com/watch?v=-4d3PgEXhdY CH32V203 Phi 3 Vision Released https://azure.microsoft.com/en-us/blog/new-models-added-to-the-phi-3-family-available-on-microsoft-azure/ Ollama ChatGPT 4o https://openai.com/index/hello-gpt-4o/ Book of the Show Patrick: MyFirstMillion Podcast https://www.mfmpod.com/ Jason: A Path Towards Autonomous Machine Intelligence https://openreview.net/pdf?id=BZ5a1r-kVsf Patreon https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Dave the Diver https://store.steampowered.com/app/1868140/DAVE_THE_DIVER/ Jason: Turing Complete https://store.steampowered.com/app/1444480/Turing_Complete/ Topic: DevOps What is DevOps DevOps vs SRE Why DevOps is important Engineering time is expensive Outages can hurt company metrics & reputation Build & Testing Infrastructure Bazel & Build/Test Idempotency Build/Test Farms BuildBarn Github Actions Jenkins Infrastructure as code Terraform Blue Green Deployment Continuous Everything! Continuous Integration Continuous Deployment How to Measure DevOps Build Times Release cadence Bug tracking / round trip times Engineer Surveys Time spent doing what you enjoy DevOps Horror Stories ★ Support this podcast on Patreon ★…

1
173: Mocking and Unit Tests 1:35:22

1 year ago1:35:22

1:35:22

173: Mocking and Unit Tests Intro topic: Headphones News/Links: Texas A&M University Physics Festival https://physicsfestival.tamu.edu/ Rust vs Cpp at Google Lars Bergstrom (Google Director of Engineering): Rust teams at Google are as productive as the ones using Go and 2x those using Cpp https://youtu.be/6mZRWFQRvmw?t=27012 Is Cosine Similarity Really About Similarity https://arxiv.org/abs/2403.05440 Xz utils supply chain attack Andres Freund at Microsoft https://arstechnica.com/security/2024/04/what-we-know-about-the-xz-utils-backdoor-that-almost-infected-the-world/ Book of the Show Patrick: 80/20 Running by Matt Fitzgerald https://amzn.to/3xyEKLo Jason: A Movie Making Nerd https://amzn.to/49ycDJj Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Shapez Android: https://play.google.com/store/apps/details?id=com.playdigious.shapez&hl=en_US&gl=US Shapez iOS: https://apps.apple.com/us/app/shapez-factory-game/id6450830779 Jason: Dwarf Fortress https://store.steampowered.com/app/975370/Dwarf_Fortress/ Topic: Mocking and Unit Tests What are Unit Tests Balance between utility, maintenance, and coverage Unit Test: testing small functions Regression Test: Testing larger functions System Test: End-to-end testing of programs What are mocks & fakes When to use mock vs. fake Mocking libraries in various languages Python: https://docs.python.org/3/library/unittest.mock.html Java: https://github.com/mockito/mockito C++: https://github.com/google/googletest ★ Support this podcast on Patreon ★…

1
172: Transformers and Large Language Models 1:26:08

1 year ago1:26:08

1:26:08

172: Transformers and Large Language Models Intro topic: Is WFH actually WFC? News/Links: Falsehoods Junior Developers Believe about Becoming Senior https://vadimkravcenko.com/shorts/falsehoods-junior-developers-believe-about-becoming-senior/ Pure Pursuit Tutorial with python code: https://wiki.purduesigbots.com/software/control-algorithms/basic-pure-pursuit Video example: https://www.youtube.com/watch?v=qYR7mmcwT2w PID without a PHD https://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdf Google releases Gemma https://blog.google/technology/developers/gemma-open-models/ Book of the Show Patrick: The Eye of the World by Robert Jordan (Wheel of Time) https://amzn.to/3uEhg6v Jason: How to Make a Video Game All By Yourself https://amzn.to/3UZtP7b Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Stadia Controller Wifi to Bluetooth Unlock https://stadia.google.com/controller/index_en_US.html Jason: FUSE and SSHFS https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh Topic: Transformers and Large Language Models How neural networks store information Latent variables Transformers Encoders & Decoders Attention Layers History RNN Vanishing Gradient Problem LSTM Short term (gradient explodes), Long term (gradient vanishes) Differentiable algebra Key-Query-Value Self Attention Self-Supervised Learning & Forward Models Human Feedback Reinforcement Learning from Human Feedback Direct Policy Optimization (Pairwise Ranking) ★ Support this podcast on Patreon ★…

1
171: Compilers and Interpreters 1:25:10

1 year ago1:25:10

1:25:10

Intro topic: Monitor setups News/Links: BlueScuti, Willis, beats Tetris https://www.youtube.com/watch?v=GuJ5UuknsHU PalWorld accused of being an AI Product https://www.forbes.com/sites/paultassi/2024/01/22/palworld-accused-of-using-genai-with-no-evidence-so-far/?sh=26a9651b4239 4 Billion if-statements to determine if a number is even or odd https://andreasjhkarlsson.github.io/jekyll/update/2023/12/27/4-billion-if-statements.html Seamless M4T https://ai.meta.com/blog/seamless-m4t/ Book of the Show Patrick: Foundation by Isaac Asimov https://amzn.to/3SrmgnP Jason: Propaganda by Edward Bernays https://amzn.to/47JUCXJ Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: The Room Game https://www.fireproofgames.com/games/the-room Jason: Incredibuild https://www.incredibuild.com/ Topic: Compilers and Interpreters (Request by Jessica W.) Machine Code Architecture Specific Assembly Single vs Two Pass Compiler High level Languages Intermediate Representation JVM ByteCode vs Machine Code for portability Scripting/Interpreters JIT Profile Guided Optimization Resources https://www.craftinginterpreters.com/ https://nandgame.com/ Turing Complete ★ Support this podcast on Patreon ★…

1
170: 2023 Holiday Special Live 1:38:34

2 years ago1:38:34

1:38:34

Predictions: Jason VR for Work Lowering AI training cost/ improved efficiency RISC-V takeoff Patrick Ai claim of AGI Ai peer reviewer Ai Video Generator More space vehicles reaching orbit Early career, finding role at FAANG, liaising vs shipping code. Startup? 3 part. 1. How and when current hype for AI will end? 2. Shape of the show 3. Upcoming in tech What are essential programmer knowledge items? CS Student, how to organize life and goals? What purpose life should serve? What kind of programmer were you in college? Happy Holidays! ★ Support this podcast on Patreon ★…

1
169: HyperLogLog 1:29:33

2 years ago1:29:33

1:29:33

Intro topic: Testing your car battery News/Links: Tech Layoffs still going on https://www.sfchronicle.com/tech/article/google-layoffs-california-companies-18465600.php Real-time dreamy Cloudscapes with Volumetric Raymarching https://blog.maximeheckel.com/posts/real-time-cloudscapes-with-volumetric-raymarching/ Robot Rascals https://en.wikipedia.org/wiki/Robot_Rascals Meta Quest 3 https://www.theverge.com/23906313/meta-quest-3-review-vr-mixed-reality-headset Book of the Show Patrick: HyperLogLog Paper https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/40671.pdf Jason: Eureka! NVIDIA Research Breakthrough Puts New Spin on Robot Learning https://blogs.nvidia.com/blog/2023/10/20/eureka-robotics-research/ Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h Tool of the Show Patrick: Techtonica: https://store.steampowered.com/app/1457320/Techtonica/ Jason: ESP32 development board: https://amzn.to/3Qpmb20 WEMOS Topic: HyperLogLog Motivation Cardinality Counting LinearCounting Hash + expectation of collision based on how full Bloom Filter LogLog Use first N bits as bucket Use max sequential 0s in each bucket Average HyperLogLog Handle empty buckets Use correction factor like linear counting for low counts (number of empty buckets) and high counts Distributing Transfer bucket counts ★ Support this podcast on Patreon ★…

1
168: Godot 1:28:34

2 years ago1:28:34