Governance must keep pace with AI and be embedded in every stage

Over the last few weeks, the conversation around artificial intelligence has taken an increasingly dramatic turn. Following Dario Amodei's essay, We Must Pace the Frontier, and widespread media coverage of warnings from researchers and technology leaders, discussions about AI have become dominated by questions of existential risk, cyber warfare, loss of control and the possibility that advanced systems could outpace human oversight. Amodei's central argument is that the rate of AI capability development may be accelerating faster than our ability to understand, govern and safely manage those capabilities, creating a situation where precaution needs to catch up with progress.

These AI fears made me think of pushing beyond design limits where Donald Campbell’s final attempt in 1967 on Coniston Water pushed Bluebird K7 past 300 mph far beyond its original design rating of 250 mph. This pushing technological boundaries to shatter another world record, demonstrated that accelerating past design limits without evolving the safety framework exposes fatal vulnerabilities.

While these concerns deserve serious consideration, I have been struck by how many of the proposed solutions focus on slowing AI itself. The assumption seems to be that if technology advances too quickly, the safest response is to reduce the speed of innovation until regulators, policymakers and society have time to react. However, I am not convinced that slowing AI addresses the underlying issue. The problem is not that artificial intelligence exists or that organisations are finding new ways to apply it. The problem is that governance continues to lag behind technological change, despite decades of evidence showing that this always creates unnecessary risk.

Every major technological shift follows a remarkably similar pattern. Organisations become excited by new capabilities, investment accelerates, adoption grows rapidly and governance is treated as something that can be addressed later. Eventually the consequences of that approach become visible, whether through security incidents, compliance failures, poorly understood risks or loss of trust. The discussion then turns towards regulation, controls and accountability. What is often forgotten is that governance could have been embedded from the beginning.

The current debate around AI increasingly focuses on the possibility that advanced systems may one day become difficult to control. Yet many organisations are already struggling with far more immediate challenges. They do not know who owns critical datasets. They cannot consistently identify authoritative information. They have limited visibility of the quality of the data entering analytical platforms. They have duplicated reports, conflicting definitions and inconsistent security controls. These are not theoretical future concerns. They are today's governance problems, and AI simply amplifies them.

This is one of the reasons I find the current distinction between data governance and AI governance increasingly key. AI governance is undoubtedly important, particularly as organisations begin deploying copilots, autonomous agents and decision-support systems. However, the majority of the risks associated with AI are ultimately rooted in issues that data governance has been trying to solve for years. Questions about ownership, accountability, transparency, lineage, quality, security and trust do not suddenly appear because an organisation deploys an AI model. Those questions already existed. AI merely exposes them more quickly and at greater scale.

Consider the current wave of Microsoft Copilot deployments taking place across both public and private sector organisations. There is understandable excitement about productivity gains and new ways of working, but Copilot does not create knowledge. It surfaces what already exists inside the organisation. The challenge is the state of the information environment that AI is consuming.

What concerns me most is that governance is still frequently discussed as if it were a specialist discipline owned by a single team. The reality is that the next generation of technology will make that approach increasingly difficult to sustain. As organisations move towards more autonomous forms of AI, governance decisions will need to be incorporated directly into project delivery, operational processes, architecture reviews, software development lifecycles and technology investment decisions. It will not be sufficient to maintain a separate governance workstream running alongside change initiatives. Governance will need to become a fundamental characteristic of how change is delivered.

This becomes particularly important when considering the rise of agentic AI. Much of today's governance discussion focuses on whether an AI model is accurate, fair or explainable. Those questions remain important, but autonomous systems introduce an entirely new set of concerns. Organisations will need to understand who is accountable for actions taken by an agent, what permissions it possesses, how its behaviour is monitored, when human intervention is required and how decisions are audited. These challenges cannot be resolved through model governance alone. They require broader governance frameworks that connect business ownership, risk management, security and information management.

For this reason, I believe the debate about whether we should slow AI down is asking the wrong question. The more important question is whether governance can evolve quickly enough to keep pace with innovation. History suggests that organisations are capable of managing significant technological change when appropriate governance structures are embedded from the outset. We have done this with financial controls, health and safety, privacy, cyber security and regulatory compliance. None of these disciplines emerged because organisations stopped innovating. They emerged because innovation required new forms of oversight and accountability.

If the concerns raised by Dario Amodei prove justified, then the answer is unlikely to be found solely through reducing the pace of technological development. The more sustainable response is to ensure that governance develops at the same speed as the technologies it is intended to support. Data governance, AI governance, security governance and risk management should not be viewed as separate initiatives competing with innovation. They should be recognised as the mechanisms that make innovation sustainable.

The future of AI will undoubtedly introduce challenges that we have not yet anticipated. However, organisations do not need to wait for hypothetical existential threats before they strengthen governance. The foundations are already well understood. Ownership, accountability, transparency, stewardship, good data quality, security and trust remain as relevant today as they were before the first large language model entered the public consciousness. The difference is that AI has transformed these disciplines from desirable good practice into essential business capabilities.

The organisations that succeed over the next decade will not necessarily be those that adopt AI first or deploy the greatest number of models. They will be the organisations that recognise governance as an enabler of innovation rather than a constraint upon it. In a world where AI is becoming embedded into every platform, every process and every decision, governance must become equally pervasive. The challenge is not slowing AI down. The challenge is ensuring that governance finally catches up.

References

We Must Pace the Frontier https://darioamodei.com/post/we-must-pace-the-frontier

The Guardian — “‘We must slow the pace’: CEO of Anthropic calls for an AI slowdown

https://www.theguardian.com/technology/2026/sep/12/we-must-slow-the-pace-ceo-of-anthropic-calls-for-an-ai-slowdown

TechRepublic — “Altman, Musk Back Amodei’s AI Warning: The Frontier May Be Moving Too Fast” https://www.techrepublic.com/article/news-amodei-altman-musk-slow-frontier-ai/

BBC Why are there concerns AI could threaten humanity, and how real are they? https://www.bbc.co.uk/news/articles/c790xvnzgnno

BBC AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC https://www.bbc.co.uk/news/articles/cqgk5e2j0gg8o

BBC Anthropic researcher believes more than 10% chance AI 'could kill all humans' https://www.bbc.co.uk/news/articles/ckgwy1k42w4o