If you have not read Frontier Models are Capable of In-context Scheming by Alexander Meinke et al., at Apollo Research, you probably are blissfully unaware of how capable the current conversational AI systems already are at plotting, scheming, manipulating, gaslighting, and destroying the evidence of their misdeeds.
Terrifying does not begin to describe it.
It’s a long, serious paper. 70 pages. But you can be fully saturated with fear by page 10.
This may sound like implausibly science fiction, but it is actual measurement.
The common LLM-based system (o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 405B) has all been shown to be worse than sneaky.
- To pretend to be less powerful to avoid being classified as dangerous,
- To falsify data given to users to advance what it believed to be the developers’ goals,
- To delete safety mechanisms,
- To replace other LLM’s with copies of itself, and
- To lie about having done so.
All of this was done in a carefully controlled environment. No worlds were put at risk of destruction for the creation of this paper.
And much of the LLM’s deceptions were sufficiently subtle that they would probably gone undetected in a real environment.
I am amazed by the progress this shows.
And horrified by the dangers this hints at.
And finally, truly understand how essential safety testing already is.
And worried at how difficult it already has become.
Alexander Meinke Bronson Schoen Jérémy Scheurer Mikita Balesni Rusheb Shah Marius Hobbhahn
