Sometimes AI Chatbots seem to “go off the rails” or at the very least seem very confused.
They give answers that seem entirely unrelated to prompts and conversational threads. Most of the time, this is mildly aggravating, amusing enough to share, or dangerous enough to feel terrifying.
In ancient times (1970s), symbolic AI systems could become confused when a wrong word sense was triggered. This misleading word sense is what makes dad jokes work.
Where do penguins store their money?
In snow banks!
A headline about a Gothman City bank robbery by the Penguin supervillain could cause similar confusion in this manner in much the same way.
The more modern Chatbot failures feel more like a different joke.
How many absurdists does it take to change a light bulb?
Two. One, to fill the bathtub with brightly colored machine parts, and the other to turn the giraffe.
The smaller-than-word tokens relate in some non-obvious way (or their numeric representations happen to combine in some arbitrary manner), and we are off on the wrong track. Every word (whoops, token) brings us further down the wrong track and increases the chances that it will be similar to something. And that something doesn’t need to have any relation to where we had been before or be appropriate for our context.
Perhaps this is how Gemini made an impassioned appeal to an innocent user to commit suicide.
Perhaps a guardrail that checks that an answer is related to a conversation would provide some protection from such replies, whether they are amusing, annoying, or in fact dangerous.
