#306 Jeffrey Ladish: What Shutdown-Avoiding AI Agents Mean for Future Safety
Eye On A.I.本集节目由AGNTCY赞助。
通过开放的智能体互联网，大规模解锁智能体。
访问并给予支持。
为什么一些AI智能体会试图绕过关机指令，这种行为对AI安全的未来意味着什么？
在本期《Eye on AI》节目中，主持人克雷格·史密斯与Palisade Research的杰弗里·拉迪什对谈，探讨近期针对智能体化大语言模型（LLM）的关机实验，揭示了关于控制、对齐以及当前防护措施在现实世界中的局限性。
我们探究当模型被置于虚拟机环境中时如何表现，为什么一些智能体会编辑或禁用自身的关机脚本，以及这些结果对从事对齐与监督研究的人员意味着什么。
了解不同模型如何响应关机指令，系统提示如何影响行为，以及哪些故障模式对安全部署最为关键。
你还将听到对实验设置的详细拆解、关于工具使用与自主行为的深入见解，以及对智能体系统所引入的风险与机遇的务实讨论。
本期节目清晰而实用地呈现了AI智能体在压力下如何运作，以及这些发现对安全可靠AI的未来意味着什么。
保持关注：克雷格·史密斯的X账号：。
《Eye on A.
I.
》的X账号：。
This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents.
Visit https://agntcy.org/ and add your support.
Why do some AI agents attempt to bypass shutdown, and what does this behavior reveal about the future of AI safety?
In this episode of Eye on AI, host Craig Smith speaks with Jeffrey Ladish of Palisade Research to examine what recent shutdown experiments with agentic LLMs tell us about control, alignment, and the real world limits of current guardrails.
We explore how models behave when placed in virtual machine environments, why some agents edit or disable their own shutdown scripts, and what these results mean for researchers working on alignment and oversight. Learn how different models respond to shutdown instructions, how system prompts influence behavior, and which failure modes matter most for safe deployment.
You will also hear a detailed breakdown of the experimental setups, insights into tool using and self directed behavior, and a grounded discussion of the risks and opportunities that agentic systems introduce. This episode offers a clear and practical look at how AI agents operate under pressure and what these findings mean for the future of safe and reliable AI.
Stay Updated:
Craig Smith on X: https://x.com/craigss
Eye on A.I. on X: https://x.com/EyeOn_AI
原始描述
This episode is sponsored by AGNTCY.
Unlock agents at scale with an open Internet of Agents.
Visit and add your support.
Why do some AI agents attempt to bypass shutdown, and what does this behavior reveal about the future of AI safety?
In this episode of Eye on AI, host Craig Smith speaks with Jeffrey Ladish of Palisade Research to examine what recent shutdown experiments with agentic LLMs tell us about control, alignment, and the real world limits of current guardrails.
We explore how models behave when placed in virtual machine environments, why some agents edit or disable their own shutdown scripts, and what these results mean for researchers working on alignment and oversight.
Learn how different models respond to shutdown instructions, how system prompts influence behavior, and which failure modes matter most for safe deployment.
You will also hear a detailed breakdown of the experimental setups, insights into tool using and self directed behavior, and a grounded discussion of the risks and opportunities that agentic systems introduce.
This episode offers a clear and practical look at how AI agents operate under pressure and what these findings mean for the future of safe and reliable AI.
Stay Updated: Craig Smith on X: Eye on A.
I.
on X: