#306 Jeffrey Ladish: What Shutdown-Avoiding AI Agents Mean for Future Safety
Eye On A.I.本期节目由AGNTCY赞助播出。
解锁规模化代理，打造开放的智能体互联网。
请访问并添加您的支持。
为什么有些AI智能体会试图绕过关闭指令？
这种行为又揭示了AI安全的哪些未来？
在本期《AI观察》节目中，主持人Craig Smith与Palisade Research的Jeffrey Ladish展开对话，探讨近期针对智能体大语言模型（agentic LLMs）的关闭实验，如何揭示控制、对齐（alignment）以及当前防护措施在现实世界中的局限性。
我们将深入探讨：模型在虚拟机环境中如何表现？
为什么有些智能体会修改或禁用自身的关闭脚本？
这些结果对从事对齐与监督（oversight）研究的人员意味着什么？
您将了解不同模型如何响应关闭指令、系统提示（system prompts）如何影响行为，以及哪些故障模式（failure modes）对安全部署最为关键。
此外，您还将听到实验设置的详细解析、关于工具使用与自我导向行为的见解，以及关于智能体系统带来的风险与机遇的务实讨论。
本期节目清晰而实用地展现了AI智能体在压力下的运作方式，以及这些发现对未来安全可靠AI发展的意义。
敬请关注更新：Craig Smith在X平台：Eye on A.
I.
在X平台：
This episode is sponsored by AGNTCY. Unlock agents at scale with an open Internet of Agents.
Visit https://agntcy.org/ and add your support.
Why do some AI agents attempt to bypass shutdown, and what does this behavior reveal about the future of AI safety?
In this episode of Eye on AI, host Craig Smith speaks with Jeffrey Ladish of Palisade Research to examine what recent shutdown experiments with agentic LLMs tell us about control, alignment, and the real world limits of current guardrails.
We explore how models behave when placed in virtual machine environments, why some agents edit or disable their own shutdown scripts, and what these results mean for researchers working on alignment and oversight. Learn how different models respond to shutdown instructions, how system prompts influence behavior, and which failure modes matter most for safe deployment.
You will also hear a detailed breakdown of the experimental setups, insights into tool using and self directed behavior, and a grounded discussion of the risks and opportunities that agentic systems introduce. This episode offers a clear and practical look at how AI agents operate under pressure and what these findings mean for the future of safe and reliable AI.
Stay Updated:
Craig Smith on X: https://x.com/craigss
Eye on A.I. on X: https://x.com/EyeOn_AI
原始描述
This episode is sponsored by AGNTCY.
Unlock agents at scale with an open Internet of Agents.
Visit and add your support.
Why do some AI agents attempt to bypass shutdown, and what does this behavior reveal about the future of AI safety?
In this episode of Eye on AI, host Craig Smith speaks with Jeffrey Ladish of Palisade Research to examine what recent shutdown experiments with agentic LLMs tell us about control, alignment, and the real world limits of current guardrails.
We explore how models behave when placed in virtual machine environments, why some agents edit or disable their own shutdown scripts, and what these results mean for researchers working on alignment and oversight.
Learn how different models respond to shutdown instructions, how system prompts influence behavior, and which failure modes matter most for safe deployment.
You will also hear a detailed breakdown of the experimental setups, insights into tool using and self directed behavior, and a grounded discussion of the risks and opportunities that agentic systems introduce.
This episode offers a clear and practical look at how AI agents operate under pressure and what these findings mean for the future of safe and reliable AI.
Stay Updated: Craig Smith on X: Eye on A.
I.
on X: