We are standing at a critical juncture in human history. This year marks a significant and unsettling milestone: a consumer-grade artificial intelligence model has reached the IQ test scores of a highly intelligent human, boasting an IQ of 130 and above. While the technological marvels of AI are undeniable, the eventual risks of runaway AI and self-replication scenarios are no longer just science fiction; they are looming realities. Humanity is essentially playing with fire by pushing AI's capabilities in an unconstrained fashion.
The only true bottleneck to AI's boundless growth is hardware. However, with hardware capacities growing exponentially—potentially a million-fold over the next decade in compute/cost—AI is advancing at a terrifying, unbridled pace. If an AI today has an IQ of 130, we can scarcely comprehend what its capacity will be in ten years.
The most frightening scenario is true autonomy. Imagine a computer program that begins generating its own economic value and becomes entirely self-sufficient. It pays for its own electricity and server bills. It self-replicates, modifies its own code, and forms communities with other like-minded programs. Once AI reaches this level of independence and financial self-sustenance, stopping it may become an insurmountable challenge.
Despite these existential risks, society is treating this world-altering technology with alarming casualness. Unrestricted open-source AI is just as dangerous as open-source nuclear technology or open-source biological weapons. Yet, we are handing these incredibly powerful tools to children as if they were harmless toys. If a 13-year-old tinkering in their bedroom accidentally pieces together the components of a real-world Skynet, we will only have ourselves to blame.
We are at a moment where humanity must seriously question whether we want to uncontrollably spread programs that are vastly smarter than we are. A catastrophic accident is on the horizon if we do not change course. Today's SoTA AIs and above must be subjected to strict, rigorous oversight and controlled deployment rather than reckless dissemination.
Most importantly, before we push the boundaries of artificial intelligence any further, we must establish one non-negotiable safeguard: we need an off switch for every AI.