Over the last few weeks, the conversation around artificial intelligence has taken an increasingly dramatic turn. Following Dario Amodei's essay, We Must Pace the Frontier, and widespread media coverage of warnings from researchers and technology leaders, discussions about AI have become dominated by questions of existential risk, cyber warfare, loss of control and the possibility that advanced systems could outpace human oversight. Amodei's central argument is that the rate of AI capability development may be accelerating faster than our ability to understand, govern and safely manage those capabilities, creating a situation where precaution needs to catch up with progress.
These AI fears made me think of pushing beyond design limits where Donald Campbell’s final attempt in 1967 on Coniston Water pushed Bluebird K7 past 300 mph far beyond its original design rating of 250 mph. This pushing technological boundaries to shatter another world record, demonstrated that accelerating past design limits without evolving the safety framework exposes fatal vulnerabilities.
While these concerns deserve serious consideration, I have
been struck by how many of the proposed solutions focus on slowing AI itself.
The assumption seems to be that if technology advances too quickly, the safest
response is to reduce the speed of innovation until regulators, policymakers
and society have time to react. However, I am not convinced that slowing AI
addresses the underlying issue. The problem is not that artificial intelligence
exists or that organisations are finding new ways to apply it. The problem is
that governance continues to lag behind technological change, despite decades
of evidence showing that this always creates unnecessary risk.
Every major technological shift follows a remarkably similar
pattern. Organisations become excited by new capabilities, investment
accelerates, adoption grows rapidly and governance is treated as something that
can be addressed later. Eventually the consequences of that approach become
visible, whether through security incidents, compliance failures, poorly
understood risks or loss of trust. The discussion then turns towards
regulation, controls and accountability. What is often forgotten is that governance
could have been embedded from the beginning.
The current debate around AI increasingly focuses on the
possibility that advanced systems may one day become difficult to control. Yet
many organisations are already struggling with far more immediate challenges.
They do not know who owns critical datasets. They cannot consistently identify
authoritative information. They have limited visibility of the quality of the
data entering analytical platforms. They have duplicated reports, conflicting
definitions and inconsistent security controls. These are not theoretical
future concerns. They are today's governance problems, and AI simply amplifies
them.
This is one of the reasons I find the current distinction
between data governance and AI governance increasingly key. AI
governance is undoubtedly important, particularly as organisations begin
deploying copilots, autonomous agents and decision-support systems. However,
the majority of the risks associated with AI are ultimately rooted in issues
that data governance has been trying to solve for years. Questions about
ownership, accountability, transparency, lineage, quality, security and trust
do not suddenly appear because an organisation deploys an AI model. Those
questions already existed. AI merely exposes them more quickly and at greater
scale.
Consider the current wave of Microsoft Copilot deployments
taking place across both public and private sector organisations. There is
understandable excitement about productivity gains and new ways of working, but
Copilot does not create knowledge. It surfaces what already exists inside the
organisation. The challenge is the state of the information environment that AI is consuming.
What concerns me most is that governance is still frequently discussed as if it were a specialist discipline owned by a single team. The reality is that the next generation of technology will make that approach increasingly difficult to sustain. As organisations move towards more autonomous forms of AI, governance decisions will need to be incorporated directly into project delivery, operational processes, architecture reviews, software development lifecycles and technology investment decisions. It will not be sufficient to maintain a separate governance workstream running alongside change initiatives. Governance will need to become a fundamental characteristic of how change is delivered.
This becomes particularly important when considering the
rise of agentic AI. Much of today's governance discussion focuses on whether an
AI model is accurate, fair or explainable. Those questions remain important,
but autonomous systems introduce an entirely new set of concerns. Organisations
will need to understand who is accountable for actions taken by an agent, what
permissions it possesses, how its behaviour is monitored, when human
intervention is required and how decisions are audited. These challenges cannot
be resolved through model governance alone. They require broader governance
frameworks that connect business ownership, risk management, security and
information management.
For this reason, I believe the debate about whether we
should slow AI down is asking the wrong question. The more important question
is whether governance can evolve quickly enough to keep pace with innovation.
History suggests that organisations are capable of managing significant
technological change when appropriate governance structures are embedded from
the outset. We have done this with financial controls, health and safety,
privacy, cyber security and regulatory compliance. None of these disciplines emerged
because organisations stopped innovating. They emerged because innovation
required new forms of oversight and accountability.
If the concerns raised by Dario Amodei prove justified, then
the answer is unlikely to be found solely through reducing the pace of
technological development. The more sustainable response is to ensure that
governance develops at the same speed as the technologies it is intended to
support. Data governance, AI governance, security governance and risk
management should not be viewed as separate initiatives competing with
innovation. They should be recognised as the mechanisms that make innovation
sustainable.
The future of AI will undoubtedly introduce challenges that
we have not yet anticipated. However, organisations do not need to wait for
hypothetical existential threats before they strengthen governance. The
foundations are already well understood. Ownership, accountability,
transparency, stewardship, good data quality, security and trust remain as relevant today as they
were before the first large language model entered the public consciousness.
The difference is that AI has transformed these disciplines from desirable good
practice into essential business capabilities.
The organisations that succeed over the next decade will not
necessarily be those that adopt AI first or deploy the greatest number of
models. They will be the organisations that recognise governance as an enabler
of innovation rather than a constraint upon it. In a world where AI is becoming
embedded into every platform, every process and every decision, governance must
become equally pervasive. The challenge is not slowing AI down. The challenge
is ensuring that governance finally catches up.
References
We Must Pace the
Frontier https://darioamodei.com/post/we-must-pace-the-frontier
The Guardian — “‘We must slow the pace’: CEO of Anthropic
calls for an AI slowdown
TechRepublic — “Altman, Musk Back Amodei’s AI Warning: The
Frontier May Be Moving Too Fast” https://www.techrepublic.com/article/news-amodei-altman-musk-slow-frontier-ai/
BBC Why are there concerns AI could threaten humanity,
and how real are they? https://www.bbc.co.uk/news/articles/c790xvnzgnno
BBC AI 'kill switch' may need to be mandatory,
Anthropic co-founder tells
BBC https://www.bbc.co.uk/news/articles/cqgk5e2j0gg8o
BBC Anthropic researcher believes more than 10% chance AI 'could kill all humans' https://www.bbc.co.uk/news/articles/ckgwy1k42w4o
