Madeline Carr, Consortium News.
A global debate that treats AI governance as an aspiration should at least recognise that one country is already ahead of the curve.
Recent warnings from American AI leaders have created the impression that the world is only beginning to confront the dangers of advanced artificial intelligence.
Dario Amodei, chief executive of tech company Anthropic, has called for governments to intervene to slow the development of increasingly powerful AI models.
His proposal begins with independent evaluators embedded inside AI companies, moves on to regulation and progresses to government-backed coordination among democratic states.
Ultimately, it seeks global agreements with China on testing of AI systems, dangerous applications of the technology and the pace of AI development.
Yet there is a China-shaped hole in this narrative. The Asian superpower is portrayed as the difficult final participant in a governance system that the U.S. and its allies must first create.
In fact, China has already established an extensive system of AI rules, assessment, filing, technical standards and government oversight. Here’s how China’s AI governance system has developed in stages alongside the technology itself.
Beijing’s 2022 rules on algorithmic recommendation services apply to technologies that select, rank, filter or recommend online content to users.
They compel companies that use algorithms to examine how they work, to protect the personal information of users, to give users certain controls, and to introduce safeguards for workers, consumers, children and older people.
China’s deep synthesis provisions, effective from January 2023, extended regulation to synthetic (or AI-generated) text, images, audio and video. AI companies must authenticate users, protect data, keep relevant records, assess their systems and identify synthetic material that could be mistaken for authentic content.
The interim measures for generative artificial intelligence services were introduced in August 2023. These added rules for public-facing generative AI, particularly those that can influence public opinion or persuade uses in other ways.
The measures apply to the legality and quality of training data, users’ personal information, intellectual property and discrimination based on protected characteristics. The rules also cover transparency by AI companies and the accuracy and reliability of products.
Under the rules, companies must address problems found in their services. These services must undergo a security assessment to examine their risks and compliance with Chinese legislation. The companies must also file information about their algorithms with Chinese authorities.
These mechanisms are not identical to the independent evaluators that would be embedded in AI companies, as has been proposed by Amodei. But they perform some of the same functions.
Frontier Risks
Beijing’s latest set of measures is called the AI safety governance framework 3.0, published in September. It sets out a broad approach to identifying, grading and responding to risks throughout the lifecycle of AI systems.
Building on earlier versions, the framework covers AI model and data risks, vulnerabilities in open source models (those which can be freely modified by users), AI’s effects on employment and society and increasingly autonomous systems.
It includes a dedicated mechanism for managing risks associated with rapidly developing AI agents (systems that carry out tasks in response to instructions by a user).
The framework addresses some of the same dangers now generating alarm in Silicon Valley, including highly autonomous systems, the misuse of chemical, biological and nuclear technologies, and the possibility of humans losing control of advanced AI. It proposes continual safety evaluations, risk based safeguards, humans being in charge and emergency intervention mechanisms.
An independent analysis by the U.S. Carnegie Endowment for International Peace think-tank has described the framework as China’s attempt to “to advance frontier safety without the need for heavy handed regulatory interventions.”
Legal Consequences
None of this means that Chinese AI rules should be transplanted elsewhere. Its institutions, regulatory traditions and technological priorities are different.
Many functions now presented as urgent aspirations in the U.S., including evaluation, accountability for AI companies, risk classification for AI systems, regulatory filing, technical standards and government oversight are already embedded in the governance architecture of the world’s other AI superpower.
The U.S. is free to conclude that a market-led system best serves its economic and strategic interests. The Trump administration has favoured a “minimally burdensome” national approach to AI and has sought to limit more demanding state-level regulation.
But a market-led model cannot mean privatising the rewards of AI while exporting its risks. U.S. companies may capture the commercial gains, while governments elsewhere are left to investigate security incidents, protect public systems and absorb the wider costs when those technologies fail. The OpenAI agent that gained unauthorised access to Australia’s state healthcare service shows how AI risks are being exported.
If Washington rejects extensive controls before systems are released, the U.S. needs a credible legal regime for assessing who is liable for damage caused by those AI models. Otherwise, countries dependent on foreign AI systems will face increasingly difficult choices about whose technology to trust, and on what terms.
Silicon Valley’s governing ethos has long been to “move fast and break things.” That was always a questionable principle for technologies woven deeply into public life. It is untenable for companies whose own leaders now warn that their products could cause catastrophic harm.
Beijing has already surrounded its AI sector with public obligations, regulatory oversight and an evolving framework of risk assessment. A global debate that treats such governance as a future aspiration should at least recognise that one country is already ahead of the curve.
Madeline Carr is professor of global politics and cybersecurity at University College London. From 2023 – 2025, she was also visiting chair of technology at Schwarzman College, Tsinghua University. Her research focuses on the implications of emerging technology for national and global security, international order, and corporate governance. Professor Carr has published on cyber norms, multi-stakeholder Internet governance, the future of the insurance sector in the IoT, cybersecurity and international law, the public/private partnership in national cyber security strategies, global technology standards, and the global governance of data as well as the ways in which boards approach cyber risk. Professor Carr is co-director of an interdisciplinary Centre for Doctoral Training (CDT) in Cybersecurity at UCL and deputy director of a second CDT on hybrid threats. From 2018 – 2022, Professor Carr was director of the U.K.-wide Research Institute for Sociotechnical Cyber Security and developed a research programme on cybersecurity in local government. Professor Carr is a senior research associate at RUSI and a fellow of the Global China Academy.
Editor’s Note: At a moment when the once vaunted model of responsible journalism is overwhelmingly the play thing of self-serving billionaires and their corporate scribes, alternatives of integrity are desperately needed, and ScheerPost is one of them. Please support our independent journalism by contributing to our online donation platform, Network for Good, or send a check to our new PO Box. We can’t thank you enough, and promise to keep bringing you this kind of vital news.
You can also make a donation to our PayPal or subscribe to our Patreon.