Four days after AI “whistleblower” Jacob Coxon announced his resignation from Anthropic on grounds that AI “could kill us all by the end of the decade”, the founders of the two leading AI Frontier Lab companies, Anthropic’s Dario Amodei and OpenAI’s Sam Altman, campaigned for global regulation that included antitrust waivers for their companies, severe restrictions on competitive Open Source models, and government power to throttle or out right shut down AI models deemed unsafe.
In the midst of that week where every news outlet and media publication was warning of the impending doom of misaligned AI, I had managed to make Happy Hour plans with some friends at a fancy hotel bar. In contrast to the previous time every news outlet was warning something could kill us all, many other people had also managed to put aside existential doom to enjoy a glass of organic, biodynamic wine — I suspect it was because this time the media didn’t tell us to stock up on eggs and toilet paper in hazmat suits and lock ourselves in quarantine.
As the founder of the Misalignment Museum, opened in 2023 to foster discourse about both the amazing potential and possible risks of AI, the concept of (p)doom is not foreign to me. This time, however, I was uniquely alarmed.
There was nothing new about the concerns raised by Jacob, Sam and Dario — these have been common technical debate topics in AI circles for years; what was unique this time was that both friends I was meeting shared that their parents (one from Sweden and one from Boston, all with no background in tech) had called them in a state of panic about the urgent need for global AI regulation in the form of the very talking points extolled by this recent media cycle. These parents who had no understanding of what Open Source and Open Weights even meant were suddenly and passionately advocating for regulation that would centralize arguably the greatest power ever created by humanity, to a vague, non-elected entity, while also checkmating the world into a system without checks and balances.
When alarms about agents breaking out of sandboxes evoke images of children playing in literal sand, and words like Anthropic, OpenAI, ChatGPT and Claude are synonymous with making cat dancing videos to probably 99% of the world, a humanity saving strategy is not to scare those people to hand the fate of the world to a handful of concerned tech people who “know better.”
Indeed, those developing frontier models are uniquely positioned to understand them, but when one “knows better”, they have a responsibility to help educate and equip as many people as they can to think critically and consciously participate in the decisions that will be impacting them. Only after a baseline of understanding is established should reasoned cases for one particular regulation or other be appealed.
The hubris of central planning seems to always lead to disaster. Smart people often fall into the trap of believing freedom is unsustainable and must be replaced with centralized control by experts. While well intentioned centralized systems sing a siren song of efficiency and optimization, it is dependent on a hierarchy that must be both infallible and benevolent.
The historically repeated danger is that these systems become blind, not omniscient, to the information and negative impact dispersed among millions of diverse, subjugated people. Systems of checks and balances enable decentralized correction, and at least a fighting chance to a progressively better world.
Even if 99% of the world opts for some proposed global AI policy, we do not have a free society if they do not understand what they are opting into.
One example that comes to mind is the biometric, facial recognition screening that TSA rolled out in airports a few years ago. Without any explicit permission or insight into how our biometric data is stored or used, airports have rolled out AI powered facial recognition software in security lines that have a small paper sized sign reading:
“Participation in TSA facial recognition technology is optional. Your photo is deleted after identity is verified. Advise the officer if you do not want your photo taken.”
What probably 99% of people who are default opted in don’t realize is that once a photo of a person is processed with AI computer vision software, the AI has a model of that person’s biometric data — it doesn’t need that specific photo to be able to recognize or identify that person in the future.
Every day, millions of people, by not proactively “opting out” while rushing to catch their flights, are giving permission to have their biometric data, linked with their IDs, travel history, and location, to be used and stored by vague private companies that are also vulnerable to hacking, misuse and data leaks. The public’s asymmetrical lack of knowledge about AI technology is being leveraged to manipulate compliance.
To be clear, I believe there is need for thoughtful AI regulation and possibly eventual global coordination — what we need first is Faustian wisdom that well intentioned directives can create the very apocalyptic outcomes we are trying to prevent, and that “freedom and life are earned by those alone who conquer them each day anew.”
In C.S. Lewis’ book, Till We Have Faces, he retells the myth of Cupid and Psyche where the beautiful princess Psyche is sacrificed to end a famine and plague. When Psyche’s older sister, Orual, returns to the site to bury her, she is shocked to find Psyche alive and seemingly well.
Psyche says a god, who only comes at night and forbids her from looking at his face, has taken her as his wife. Orual cannot see the luxurious palace and nourishing food of the god realm Psyche now lives in, and is therefore convinced her sister is prey to a monstrous bandit. She urges Psyche to disobey her husband and hold a lamp to his face to know Orual’s “truth” that he is not a god, but some sort of monster.
When Psyche refuses, Orual stabs her own arm and threatens to kill Psyche and then herself unless she complies. Terrorized by Orual’s deformed love and care, and in fear that not complying will ‘kill us all’, Psyche agrees. That night when she holds the lamp to see her husband Cupid’s beautiful face, she is exiled and condemned to laborious wandering and suffering.
Later, Orual feels shame for violating her teacher’s wisdom: “One of his maxims was that if we cannot persuade our friends by reason we must be content ‘and not bring a mercenary army to our aid.’ (He meant passions.)” Orual weaponized Psyche’s love for her, and operated with the hubris that she knew better than Psyche knew for herself what was good for her.
Upon Psyche’s banishment, the gods condemned Orual too with the curse: “You also shall be Psyche.”
I believe many of those calling for globally centralized AI regulations, like Orual to Psyche, have love for humanity and I believe they have noble intentions, but the danger lies in love deformed by possession — the impassioned despotism and hubris of omniscience, thinking they know better what is good for a person without engaging the person themselves, without knowing the faces of the very people they are trying to save.
At this moment, more imminent than risk of hypothetical AI assisted bioweapons and Open Weight models is checkmating the world into a totalitarian monopoly where educated and individually reasoned freedoms are abdicated, where we do not consider the faces of the individuals who comprise humanity, and a means of future recourse or defense against hypothetical risks is made impotent. Achieving compliance without understanding is manipulation.
After Psyche’s banishment, as penance she is forced by the goddess Ungit to complete tasks impossible for an individual to complete — a metaphor for a type of distributed engagement: the first to sort a massive mound of barley, millet, poppy, lentils and beans before the evening; second, to collect the golden wool from a fierce sheep; third to fetch water from a deadly height.
Psych is helped by ants to complete the sorting of seeds. She is able to collect the golden wool snagged on bushes. An eagle helps carry the bowl of water from the precipitous mountain.
No single person or group of people has divine omniscience to know what is best for everyone, no matter how smart, and neither will any small group of people have the omnipotence to produce a living outcome. As Psyche needs the help of the ants, the bushes and the eagle to survive, any viable path forward for humanity requires shared participation.
AI Safetyism needs to start with the belief that people have the capacity to understand the world they live in, and that they have the dignity and responsibility to participate in the actions and decisions that will create the future.
—-
The mission of Misalignment Museum is expanding knowledge of AI for the purpose of elevating public discourse to equip people to participate in building the vibrant and hopeful future that is possible.
