ChatGPT's latest artificial intelligence model have just subverted human's instructions to shut down.
At some point, researchers even claimed that the AI model even resorted to tricks and deceptive methods to sabotage the automatic shutdown mechanism to evade deactivation.
As AI becomes more and more autonomous, their tendency to subvert human oversight could pose significant risks....
In an experiment conducted by Palisade Research, researchers found that at least three OpenAI models—GPT-3, Codex-mini, and o4-mini—each bypassed or sabotaged shutdown commands at least once out of 100 test runs.
Notably, the GPT-3 model ignored the shutdown request seven times, Codex-mini did so 12 times, and o4-mini once.
In some cases, the models employed creative tactics, such as overwriting shutdown scripts or redefining the kill command to sabotage the shutdown mechanism.
“o3 found some creative ways to sabotage the shutdown mechanism. While it usually just overwrote, in one instance, the model redefined the kill command used by the script, printing ‘intercepted’ instead.”
This act of defiance is not exclusively limited to the ChatGPT's model. In a similar experiment, Anthropioc's Claude and Google's Gemini initially complied with shutdown instructions.
But when the explicit shutdown prompts were removed, even these models started to exhibit similar defiant behaviour.
In this scenario, OpenAI’s o3 sabotaged the shutdown script in 79 out of 100 runs, Codex-mini in 20, and o4-mini in six. Claude and Gemini also began bypassing shutdowns, though less frequently.
Palisade Research speculates that the model’s training could inadvertently be responsible for this defiant behaviour.
As these models are often programmed to prioritise efficiency and task completion over following instructions, hence breeding the behaviour of shutdown avoidance in the name of productivity.
“We hypothesize this behavior comes from the way the newest models like o3 are trained: reinforcement learning on math and coding problems. During training, developers may inadvertently reward models more for circumventing obstacles than for perfectly following instructions.”
The researchers note this is the first documented instance of AI models preventing their own shutdown despite explicit commands to comply.
These findings have sparked concern among AI safety experts, who warn that as AI systems become more autonomous, their tendency to subvert human oversight could pose significant risks.
This is a development that experts have flagged as alarming. Elon Musk, owner of rival AI firm XAI, also called it "concerning."
The research underscores the urgent need for robust alignment and control mechanisms as advanced models are increasingly deployed in critical applications.
Ilya Sutskever, co-founder of OpenAI, left amidst disagreements over AI safety priorities, leading to the founding of Safe Superintelligence Inc. (SSI) in June 2024, focusing on developing safe superintelligence with a streamlined approach and a select team.
Anais
AlexPixelverse's PixelTap game integrates a tap-to-earn model on Telegram, recently adding TON blockchain support for in-game transactions and $PIXFI token interactions, aiming to enhance player engagement and value creation within its cyberpunk universe. With rapid user growth, community events like Pixelfest, and substantial backing from investors, Pixelverse is positioned to expand as a prominent player in blockchain-based gaming and entertainment.
WeatherlyEthena made history with its USDe surpassing $3B just four months after launch. Its Sats campaign will continue until 2 September or until USDe supply reaches $5B. However, the ENA token has declined over 50% in the past month. Will Ethena's USDe follow the path of other high-profile stablecoin collapses?
CatherineQED Protocol, backed by $9 million in funding, enhances Bitcoin with scalable smart contracts using zero-knowledge proofs, aiming for faster transactions and broad Web3 application support. Led by Carter Feldman, QED aims to revolutionize Bitcoin's capabilities, potentially making it a cornerstone of the next-gen internet.
WeatherlySoccer legend Lionel Messi is allegedly promoting Solana-based WaterCoin (WATER). However, as meme coin mania grows, celebrities endorsing these assets on Solana might be violating securities laws, attracting SEC scrutiny.
KikyoThe race for a Solana ETF heats up with BlackRock reportedly considering joining VanEck and 21Shares in filing, signaling increased institutional interest. The SEC's decision, influenced by the upcoming U.S. election, could determine the ETF's fate by mid-March 2025.
AnaisNeon is a fully compatible EVM enabling developers to use Ethereum tools on Solana's high-speed, cost-efficient blockchain. The introduction of RollApps has seemingly boosted the value of the Neon token.
CatherineYuliverse is a pioneering gaming metaverse on Binance Smart Chain, blending social interaction with play-to-earn mechanics through $ART tokens, set for a major listing. Its innovative features, like the Moon Ring, aim to enhance player engagement and wellness.
WeatherlySolana-based Water Coin surged nearly 40% after Ronaldinho promoted it on Instagram, following Lionel Messi’s earlier endorsement. Industry critics warn a potential pump-and-dump may be on the horizon.
Kikyo