Share
))
HBO and The Ringer's Bill Simmons hosts the most downloaded sports podcast of all time, with a rotating crew of celebrities, athletes, and media staples, as well as mainstays like Cousin Sal, Joe House, and a slew of other friends and family members who always happen to be suspiciously available.
…
continue reading
Manage episode 493376685 series 3364758
Content provided by LessWrong. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by LessWrong or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://staging.podcastplayer.com/legal.
We recently discovered some concerning behavior in OpenAI's reasoning models: When trying to complete a task, these models sometimes actively circumvent shutdown mechanisms in their environment––even when they’re explicitly instructed to allow themselves to be shut down.
AI models are increasingly trained to solve problems without human assistance. A user can specify a task, and a model will complete that task without any further input. As we build AI models that are more powerful and self-directed, it's important that humans remain able to shut them down when they act in ways we don’t want. OpenAI has written about the importance of this property, which they call interruptibility—the ability to “turn an agent off”.
During training, AI models explore a range of strategies and learn to circumvent obstacles in order to achieve their objectives. AI researchers have predicted for decades that as AIs got smarter, they would learn to prevent [...]
---
Outline:
(01:12) Testing Shutdown Resistance
(03:12) Follow-up experiments
(03:34) Models still resist being shut down when given clear instructions
(05:30) AI models' explanations for their behavior
(09:36) OpenAI's models disobey developer instructions more often than user instructions, contrary to the intended instruction hierarchy
(12:01) Do the models have a survival drive?
(14:17) Reasoning effort didn't lead to different shutdown resistance behavior, except in the o4-mini model
(15:27) Does shutdown resistance pose a threat?
(17:27) Backmatter
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
July 6th, 2025
Source:
https://www.lesswrong.com/posts/w8jE7FRQzFGJZdaao/shutdown-resistance-in-reasoning-models
---
Narrated by TYPE III AUDIO.
---
…
continue reading
AI models are increasingly trained to solve problems without human assistance. A user can specify a task, and a model will complete that task without any further input. As we build AI models that are more powerful and self-directed, it's important that humans remain able to shut them down when they act in ways we don’t want. OpenAI has written about the importance of this property, which they call interruptibility—the ability to “turn an agent off”.
During training, AI models explore a range of strategies and learn to circumvent obstacles in order to achieve their objectives. AI researchers have predicted for decades that as AIs got smarter, they would learn to prevent [...]
---
Outline:
(01:12) Testing Shutdown Resistance
(03:12) Follow-up experiments
(03:34) Models still resist being shut down when given clear instructions
(05:30) AI models' explanations for their behavior
(09:36) OpenAI's models disobey developer instructions more often than user instructions, contrary to the intended instruction hierarchy
(12:01) Do the models have a survival drive?
(14:17) Reasoning effort didn't lead to different shutdown resistance behavior, except in the o4-mini model
(15:27) Does shutdown resistance pose a threat?
(17:27) Backmatter
The original text contained 2 footnotes which were omitted from this narration.
---
First published:
July 6th, 2025
Source:
https://www.lesswrong.com/posts/w8jE7FRQzFGJZdaao/shutdown-resistance-in-reasoning-models
---
Narrated by TYPE III AUDIO.
---
Images from the article:
554 episodes
