Expert Comment: AI’s existential crisis
Brent Mittelstadt, Professor of Data Ethics and Policy, and Sandra Wachter, Professor of Technology and Regulation, both at the Oxford Internet Institute, argue that as AI companies warn of the existential risks of their own technology, the public that funds and fuels it should demand more than promises of economic growth.
Artificial intelligence (AI) is having an identity crisis. How can a technology simultaneously be celebrated as one of humanity’s most promising inventions, while also allegedly being on the cusp of destroying us all?
This paradox is not new. AI companies and researchers have been boasting about the power and promise of their creation for many years. Critics have also rightly warned about the many risks posed by AI, ranging from the perpetuation of harmful biases and theft of intellectual property and labour, through to the technology’s significant environmental impact and risk of job losses.
But a shift in the tenor of public debate has been notable in recent weeks. Among AI proponents, a new consensus has started to emerge: the most capable AI models (or ‘frontier AI’) are highly risky, unpredictable, and pose existential threats to humanity that must be controlled.
Worries abound over runaway ‘artificial superintelligence’, or models that outperform humans on all types of tasks requiring intelligence, being able to autonomously improve themselves faster than humanity can keep up. Some have compared the risks posed by AI to nuclear weapons. CEOs of major AI companies including Anthropic, Google, and OpenAI are now calling for global cooperation and a slowdown of development on the most capable AI models, or ‘frontier AI’, to ensure the technology is built and used safely and responsibly.
This shift in narrative follows a series of incidents recently reported by companies themselves. These reports allege that AI is now so powerful and unpredictable that it can break loose from human control, including escaping testing environments or ‘sandboxes’, hacking websites and online platforms, and engaging in other risky ‘emergent’ or non-programmed behaviours.
“Given all these warnings, one could rightly ask: why are companies racing to build a technology that they themselves describe as a doomsday device? If the technology seriously poses such severe risks, why continue to develop it?”
Given all these warnings, one could rightly ask: why are companies racing to build a technology that they themselves describe as a doomsday device? If the technology seriously poses such severe risks, why continue to develop it?
This is where the duality of AI comes into full view. The logic goes like this: AI is high risk, but also high reward. According to its proponents, AI will be extensively adopted across society and drive massive economic benefits through augmentation and replacement of human intellectual labour. Failing to develop frontier AI as quickly as possible risks being left behind and losing out to global competitors.
AI and the public
Where the public fit into this equation remains unclear. In theory, economic benefits broadly benefit society in theory. In practice, history teaches us that ‘boom and bust’ tech cycles, like the one we’re currently going through with AI, rarely bring socialised gains and privatised losses; quite the opposite, in fact. Even if AI proves to be wildly economically beneficial, it may come at the cost of human jobs, opportunities, and economic equality.
It would seem then that, in either case, the public loses. Either AI destroys us all in spectacular fashion, or it degrades our work and well-being to the point where it may as well have destroyed humanity.
“It would seem then that, in either case, the public loses. Either AI destroys us all in spectacular fashion, or it degrades our work and well-being to the point where it may as well have destroyed humanity. ”
However, the public is listening. Unease about AI is growing, seen in recent student movements and protests against the construction of data centres in the USA, bans on generative AI in schools, and resistance movements such as QuitGPT.
At the same time, governments and policymakers have continued to subsidise and lower barriers to AI development through deregulation. AI companies enjoy public funding and tax breaks, lax enforcement of existing regulations, as well as privileged access to land, water, energy, and public infrastructure. Frontier AI systems are likewise built upon public data and the collective creative works of humanity. The public rarely has a say in these arrangements, despite ultimately paying for them.
Getting a fair share
The public should reap the benefits of the AI boom. People are shareholders in frontier AI in all but name and should demand technology that improves their lives in return.
“The public should reap the benefits of the AI boom. People are shareholders in frontier AI in all but name and should demand technology that improves their lives in return.”
Technology policy can provide a path for change. Policymakers are public servants. They have a mandate to protect the common good which is currently being threatened by frontier AI.
In the UK and Europe, technology policy currently seeks to balance two aims: economic growth and risk mitigation. The basic idea is that we should use policy to support the building of technologies that economically benefit society while posing the fewest risks possible. When evaluating how public resources should be used to support AI development, policy-makers have tended to focus primarily on economic growth and mitigating risks.
Thinking of technology policy in this way is rather limiting. Economic growth is only one way in which technology can benefit society. Seeking to minimise risk, while sensible at first glance, is an inherently reactive goal. Technologies that pose minimal risks do not necessarily benefit society.
“Policymakers are public servants. They have a mandate to protect the common good which is currently being threatened by frontier AI.”
Happiness as a tech policy goal
A change of priorities is urgently needed in technology policy across Europe. We need a reformulation of policy in more optimistic terms. AI companies benefiting from public support should be required to give more than vague promises about broad economic benefits. Rather, they should be required to show their technologies will concretely improve people’s happiness and well-being and be held to their promises in the long-term.
“AI companies benefiting from public support should be required to give more than vague promises about broad economic benefits. Rather, they should be required to show their technologies will concretely improve people’s happiness and well-being and be held to their promises in the long-term.”
This is not a vague or lofty policy goal. Psychological research can tell us in concrete, measurable terms what makes people happy. Decades of research point towards three reliable sources: relatedness, competence and autonomy. We can use these concepts as a framework to assess whether technologies make a positive contribution to people’s lives.
This proposal is driven by a very simple idea: technology that could not exist without people should also benefit them. AI hype is currently leading society in very dark directions. Its future rests with a few very powerful companies.
“Technology that could not exist without people should also benefit them.”
In our latest paper published in Royal Society Open Science, 'Optimizing for happiness in EU technology policy to repay public subsidies for AI', we argue that to counter this, we urgently need a broadly endorsed positive vision of AI’s intended role in society. Frontier AI development needs to be driven by a holistic view of societal progress, not just economic or security interests. Happiness as a policy goal is a step in this direction.
For more information about this story or republishing this content, please contact [email protected]
