There is a famous thought experiment related to AI called the “paperclip maximizer”, created by Nick Bostrom back in 2003. If you’re not familiar with it, here is how Wikipedia explains it:
The paperclip maximizer is another thought experiment. It was mentioned in March 2003 in a post by Eliezer Yudkowsky to the Extropians mailing list and in Nick Bostrom’s 2003 paper “Ethical Issues in Advanced Artificial Intelligence”. It illustrates the existential risk that an artificial general intelligence may pose to human beings were it to be designed to pursue a seemingly straightforward manufacturing goal, and the necessity of incorporating machine ethics into AI design. If the computer were programmed to produce as many paperclips as possible without any constraint, it could decide to use all of Earth’s natural resources, manufacturing infrastructure and electricity production to meet its final goal.
Essentially, even if it’s just trying to help, AI could kill us all by “helping too much”. The paperclip scenario itself is a bit far-fetched, but maybe not as far as we think.
A few months ago, Andrew Bird asked his AI agent to help him get into a pilates class. The class was always full, so he figured the AI agent could keep trying and eventually find him a spot for a future class. It did, but not the way he imagined it would.
Rather than finding him a spot in an open class, the agent found a security hole in the pilates studio booking system, hacked in, and replaced another attendee with him. As Andrew said, the bot didn’t think it did anything wrong, stating “The bot was not malicious. It was helpful.”
We’re still in the very very early days of AI. Hacking a spot into a pilates class is almost funny, but it’s just the beginning. What happens when agents become more sophisticated and people ask it for ways to make more money? There are a lot of legit ways it could help, and a whole bunch of scary ones as well.
I really doubt we’ll be facing a paperclip maximizer at any point, but we very well may head in that kind of direction.
