172: Transformers And Large Language Models Programming Throwdown podcast

172: Transformers and Large Language Models

7,601 subscribers

published 2y ago

Content provided by Patrick Wheeler and Jason Gauci, Patrick Wheeler, and Jason Gauci. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Patrick Wheeler and Jason Gauci, Patrick Wheeler, and Jason Gauci or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.

172: Transformers and Large Language Models

Intro topic: Is WFH actually WFC?

News/Links:

Falsehoods Junior Developers Believe about Becoming Senior
- https://vadimkravcenko.com/shorts/falsehoods-junior-developers-believe-about-becoming-senior/
Pure Pursuit
- Tutorial with python code: https://wiki.purduesigbots.com/software/control-algorithms/basic-pure-pursuit
- Video example: https://www.youtube.com/watch?v=qYR7mmcwT2w
PID without a PHD
- https://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdf
Google releases Gemma
- https://blog.google/technology/developers/gemma-open-models/

Book of the Show

Patrick: The Eye of the World by Robert Jordan (Wheel of Time)
- https://amzn.to/3uEhg6v
Jason: How to Make a Video Game All By Yourself
- https://amzn.to/3UZtP7b

Patreon Plug https://www.patreon.com/programmingthrowdown?ty=h

Tool of the Show

Patrick: Stadia Controller Wifi to Bluetooth Unlock
- https://stadia.google.com/controller/index_en_US.html
Jason: FUSE and SSHFS
- https://www.digitalocean.com/community/tutorials/how-to-use-sshfs-to-mount-remote-file-systems-over-ssh

Topic: Transformers and Large Language Models

How neural networks store information
- Latent variables
Transformers
- Encoders & Decoders
Attention Layers
- History
  - RNN
    - Vanishing Gradient Problem
  - LSTM
    - Short term (gradient explodes), Long term (gradient vanishes)
- Differentiable algebra
- Key-Query-Value
- Self Attention
Self-Supervised Learning & Forward Models
Human Feedback
- Reinforcement Learning from Human Feedback
- Direct Policy Optimization (Pairwise Ranking)

★ Support this podcast on Patreon ★

186 episodes

#Programming #Software Development #Programming Education #Tech Education #Java #Python #Patrick Wheeler and Jason Gauci #Jason Gauci #Patrick Wheeler #Tech #Podcasting Education #News #Tech News #Programming Language #Objective-c

All episodes

Top Podcasts

1
The Bill Simmons Podcast

The Ringer

113k

Unsubscribe

15h ago15h ago

Unsubscribe

Weekly+

HBO and The Ringer's Bill Simmons hosts the most downloaded sports podcast of all time, with a rotating crew of celebrities, athletes, and media staples, as well as mainstays like Cousin Sal, Joe House, and a slew of other friends and family members who always happen to be suspiciously available.

1
PTI

ESPN, Tony Kornheiser, Michael Wilbon

105k

200

Unsubscribe

3d ago3d ago

Unsubscribe

Weekly+

Tony Kornheiser and Michael Wilbon face off in the nation's capital on the day's hottest topics.

1
The Herd with Colin Cowherd

iHeartPodcasts and The Volume

12k

Unsubscribe

17h ago17h ago

Unsubscribe

Daily+

The Herd with Colin Cowherd is a thought-provoking, opinionated, and topic-driven journey through the top sports stories of the day.

1
Planet Money

NPR

192k

355

Unsubscribe

3d ago3d ago

Unsubscribe

Weekly+

Wanna see a trick? Give us any topic and we can tie it back to the economy. At Planet Money, we explore the forces that shape our lives and bring you along for the ride. Don't just understand the economy – understand the world. Wanna go deeper? Subscribe to Planet Money+ and get sponsor-free episodes of Planet Money, The Indicator, and Planet Money Summer School. Plus access to bonus content. It's a new way to support the show you love. Learn more at plus.npr.org/planetmoney

1
Comedy of the Week

BBC Radio 4

166k

Unsubscribe

1d ago1d ago

Unsubscribe

Weekly

Brighten your week with the latest BBC Radio 4 comedy.

1
The Bugle

The Bugle

124k

678

Unsubscribe

8d ago1w ago

Unsubscribe

Weekly

It's the trans-global satiricast that leaves no hot potato unbuttered. Andy Zaltzman breaks down the news with comedians from across the world including Alice Fraser, Hari Kondabolu, Chris Addison, John Oliver, Nish Kumar, Tiff Stevenson and Helen Zaltzman. Go to TheBuglePodcast.com to become a premium subscriber and get exclusive shows. Follow us on YouTube. Hosted on Acast. See acast.com/privacy for more information.

1
How Did This Get Made?

Earwolf and Paul Scheer, June Diane Raphael, Jason Mantzoukas

200k

317

Unsubscribe

4d ago4d ago

Unsubscribe

Weekly+

The award-winning comedy podcast that celebrates bad movies. Comedians and actors Paul Scheer (The League), June Diane Raphael (Grace and Frankie), and Jason Mantzoukas (Big Mouth) break down the very best of the worst films ever made—we’re talkin’ blockbuster flops, cheesy 80s action movies, Lifetime thrillers, obscure cult classics, and if we’re honest… most Nic Cage and Jason Statham movies. Plus, sometimes they’re even joined by hilarious guests like Seth Rogen, Conan O’Brien, Amy Schume ...

1
Slate Culture

Slate Podcasts

160k

Unsubscribe

1d ago1d ago

Unsubscribe

Daily

Get the Culture Gabfest and all of Slate's culture coverage here.

1
TED Talks Daily

TED

1360k

Unsubscribe

1d ago1d ago

Unsubscribe

Daily

Want TED Talks on the go? Everyday, this feed brings you our latest talks in audio format. Hear thought-provoking ideas on every subject imaginable – from Artificial Intelligence to Zoology, and everything in between – given by the world's leading thinkers and doers. This collection of talks, given at TED and TEDx conferences around the globe, is also available in video format. Hosted on Acast. See acast.com/privacy for more information.

1
NBC Nightly News with Tom Llamas

NBC News

1215k

763

Unsubscribe

1d ago1d ago

Unsubscribe

Daily

Listen to "NBC Nightly News," providing reports and analysis of the day's most newsworthy national and international events. This audio podcast, updated each evening, brings you the day's show in its entirety. For more from "Nightly News", visit NBCNightlyNews.com.

1
CBS News Roundup

CBS News

1145k

Unsubscribe

3h ago3h ago

Unsubscribe

Daily

The CBS News team wraps up the major headlines you need to know every day on the CBS News Roundup podcast. On weekday mornings, Steve Kathan delivers the “World News Roundup” and every evening you can catch up on all the day's news with Jennifer Keiper on the “World News Roundup: Late Edition”. Then, every weekend the CBS News team in Washington goes deep into the major stories on “Weekend Roundup'' hosted by Allison Keyes. Each episode features a “Kaleidoscope” segment that takes on social ...

1
Daily Boost - Motivation and Coaching

Scott Smith - Motivation and Coaching

383k

Unsubscribe

15h ago15h ago

Unsubscribe

Daily

Start your day with practical philosophy that actually works. In under 10 minutes, Scott Smith cuts through the noise with straight talk about navigating life's messy middle—where personal and professional challenges collide. No hype. No fluff. Just 20 years of hard-won wisdom, Stoic principles, and real stories from someone who's been there. Monday through Friday, learn to think clearly, act wisely, and build a life that's actually yours.

1
Radiolab

WNYC Studios

295k

630

Unsubscribe

3d ago3d ago

Unsubscribe

Weekly

Radiolab is on a curiosity bender. We ask deep questions and use investigative journalism to get the answers. A given episode might whirl you through science, legal history, and into the home of someone halfway across the world. The show is known for innovative sound design, smashing information into music. It is hosted by Lulu Miller and Latif Nasser.

1
Science Friday

Science Friday and WNYC Studios

189k

Unsubscribe

15h ago15h ago

Unsubscribe

Daily

Covering the outer reaches of space to the tiniest microbes in our bodies, Science Friday is the source for entertaining and educational stories about science, technology, and other cool stuff.

1
This American Life

This American Life

365k

Unsubscribe

1d ago1d ago

Unsubscribe

Monthly

Each week we choose a theme. Then anything can happen. This American Life is true stories that unfold like little movies for radio. Personal stories with funny moments, big feelings, and surprising plot twists. Newsy stories that try to capture what it’s like to be alive right now. It’s the most popular weekly podcast in the world, and winner of the first ever Pulitzer Prize for a radio show or podcast. Hosted by Ira Glass and produced in collaboration with WBEZ Chicago.

1
Snap Judgment

Snap Judgment and PRX

321k

504

Unsubscribe

5d ago5d ago

Unsubscribe

Weekly

Snap Judgment mixes real stories with killer beats to produce cinematic, dramatic radio. Snap’s raw, musical brand of storytelling dares listeners to see the world through the eyes of another. It's storytelling... with a BEAT.

1
Criminal

Vox Media Podcast Network

260k

359

Unsubscribe

4d ago4d ago

Unsubscribe

Weekly

Criminal is the first of its kind. A show about people who’ve done wrong, been wronged, or gotten caught somewhere in the middle. Hosted by Phoebe Judge. Named a Best Podcast of 2023 by the New York Times. Part of the Vox Media Podcast Network.

1
Sword and Scale

Sword and Scale

60k

265

Unsubscribe

3d ago3d ago

Unsubscribe

Monthly+

Sword and Scale is a weekly true crime podcast covering the dark underworld of crime and the criminal justice system’s response to it. The first episode launched January 1st, 2014 and feature stories of murder, abduction, rape, and even more bizarre forms of crime. It’s the purest form of true-crime where the raw uncensored audio tells the story. Everything from 911 calls to court testimony, interviews with victims and sometimes with perpetrators give listeners a 360 degree look at the seedy ...

1
In The Dark

The New Yorker

53k

Unsubscribe

21d ago2w ago

Unsubscribe

Monthly

In the Dark, hosted by Madeleine Baran, is an award-winning investigative-journalism podcast that started in 2016. Its first season looked at the mysterious abduction of Jacob Wetterling in rural Minnesota and the lack of accountability that sheriffs face when they fail to solve cases. Season 2 examined the case of Curtis Flowers, who was tried six times for the same crime. In 2020, In the Dark released a special report on the coronavirus pandemic in the Mississippi Delta. In 2023, In the Da ...

172: Transformers and Large Language Models

Similar to Programming Throwdown

Download the best offline podcast player

Top Podcasts