Big Data London emerging insights

It was great to be at BigDataLDN this week and see the changing landscape. 40% of all sessions had AI in the title. This is a significant shift compared to pre 2023 programmes, where AI titled sessions were typically less than 15%.

The 2026 landscape highlighted a number of shifts.

AI is no longer a track, it is the spine of the conference embedded across every discipline.

Governance and trust are now centre stage. Multiple sessions explicitly focused on trusted AI foundations, AI governance maturity, scaling AI safely, data governance as the prerequisite for AI and observability for AI systems.

Agentic AI is emerging as a major theme. This is the first year agentic AI had a dedicated theatre signalling a shift from the previous experimentation phase to operationalisation.

AI plus organisational transformation is the new battleground asking questions like how do we scale AI safely,  modernise legacy environments,  demonstrate measurable business value, and improve governance without slowing innovation.  This is a change from asking which model to use towards to how do we run an AI enabled organisation. 

Data governance is finally being treated as strategic and it was the strongest governance representation BigDataLDN has ever had.

AI for societal impact is gaining traction with talks now covering climate change and humanitarian impact broadening AI use beyond commercial use cases.

The keynotes were AI centric which created a conference theme of AI first.

I think the BigDataLDN conference has quietly repositioned itself an AI governance, AI strategy, AI engineering and AI transformation conference. 

AI governance seems like the new cloud migration in that it requires a change of mindset. The agenda showed a pattern where

• AI adoption is assumed. 
• AI scaling is the challenge. 
• Governance is the bottleneck. 
• Data foundations are the dependency. 
• Operating models are the differentiator

The conference was packed with people and exhibitors and many people spent their time queueing to get into rooms to get to listen to talks.

 

The Hierarchy of AI Oversight

Organisations frequently treat Responsible AI, AI Governance, and Data Governance as synonymous concepts. In practice, they represent three distinct, interdependent structural tiers. Treating them as interchangeable obscures how AI systems are built, verified, and operationalised within an enterprise.

A useful way to conceptualise this structure is through a three-part stack:

•  Responsible AI defines organizational intent and boundaries.
•  AI Governance establishes operational execution and control mechanisms.
•  Data Governance manages the underlying assets and pipeline inputs.

When any single tier is neglected, the entire oversight framework becomes ineffective.

Responsible AI: Establishing Strategic Intent

Responsible AI sits at the top of the stack as an explicit declaration of intent. It articulates an organisation's risk tolerance, core values, and societal commitments regarding automated systems.

This layer does not detail specific technical configurations or workflow steps. Instead, it defines the overarching ethical perimeter, addressing core themes such as non-discrimination, explainability, safety, and accountability.

Key questions addressed at this layer include:

•  What operational boundaries define acceptable versus unacceptable AI deployments?
•  What specific harms must system designs actively prevent?
•  What baseline commitments are required for external stakeholders and regulatory bodies?

While Responsible AI acts as the strategic compass, policy statements alone do not alter system behavior. Without operational enforcement, policy declarations remain purely symbolic. Operationalising these policies requires the secondary layer: AI Governance.

AI Governance: Implementing Operational Control

AI Governance provides the operational apparatus required to enforce Responsible AI policies. It consists of the decision rights, verification protocols, audit trails, and risk taxonomies that manage an AI model across its complete lifecycle.

This tier shifts abstract commitments into concrete engineering and management workflows. It covers model validation standards, change management, automated drift detection, and post-deployment monitoring. Systems like the GRAICE framework operate within this domain to standardise evaluation criteria.

Key questions addressed at this layer include:

• Which roles hold approval authority at distinct stages of model development?
• What quantitative evidence is required prior to production deployment?
• How are performance degradation, bias drift, and unexpected edge cases detected and remediated?
• What specific conditions trigger a mandatory model recall or pause?

AI Governance ensures that models operate within defined parameters over time. However, governance controls cannot ensure model integrity if the underlying inputs are flawed. Control frameworks require verifiable data inputs, which depends entirely on the foundational layer.

Data Governance: Securing the System Inputs

Data Governance manages the quality, legal basis, security, and lineage of the data fed into machine learning pipelines. Because statistical models reflect the characteristics of their training data, AI performance is constrained by the quality of its underlying data architecture.

Without robust data management, model output becomes inherently unpredictable. Issues such as unverified data sources, unrecorded pipeline transformations, or demographic skew directly compromise model outputs regardless of how stringent the AI control checks are.

Key questions addressed at this layer include:

•  What is the precise lineage and chain of custody for training and validation datasets?
•  Do clear usage rights, legal bases, and consent frameworks exist for the ingested data?
•  Is the dataset representative, accurate, and properly versioned?
•  How are data access controls and privacy-preserving techniques maintained through the pipeline?

Strong Data Governance provides the verifiable evidence base that AI Governance relies on. Without it, validation processes lack technical substance

.

Structural Pitfalls of Top-Down Implementation

A common failure mode occurs when organisations implement oversight from the top down. Leadership teams often publish high-level ethical guidelines and establish oversight committees before building the necessary operational controls or securing data infrastructure.

This top-down approach creates several operational vulnerabilities:

•  Oversight committees evaluate systems without reliable technical lineage or performance data.
•  Data quality defects and unverified assumptions are identified late in production rather than during ingestion.
•  Ambiguity surrounds technical accountability when failures occur.
• Defining ethical principles without establishing underlying governance frameworks leads to superficial compliance—where policy commitments exist on paper but cannot be verified or enforced at the engineering level.

Building a Cohesive Oversight Framework

Establishing an effective oversight framework requires starting from foundational technical controls and building upwards:

• Establish Data Integrity: Secure data lineage, document legal rights, enforce validation checks, and maintain clear data stewardship across all pipelines.
• Deploy Control Architectures: Implement repeatable stage-gate approvals, continuous testing protocols, risk logging, and lifecycle monitoring.
• Align Operational Controls with Policy Boundaries: Connect technical metrics and threshold alerts directly to high-level organizational principles and regulatory requirements.

Aligning these three disciplines transforms AI oversight from a collection of isolated policies into an integrated operational capability.

Anthropic training programmes

Over the last week I have completed three Anthropic training programmes to expand my use of AI tools from Copilot and Gemini. While the courses focus on Claude, the value goes far beyond learning a particular AI tool. I completed:

• Claude 101
• AI Fluency Framework & Foundations
• Claude Code in Action

Claude 101 provides a solid grounding in how to work effectively with large language models. It covers prompt design, structuring requests, and understanding where AI can genuinely add value versus where human judgement remains essential.

AI Fluency Framework & Foundations takes a broader view. Rather than concentrating on technology alone, it explores how individuals and organisations can develop the skills, mindset and practices needed to adopt AI successfully. It reinforces an important lesson about becoming AI-enabled is as much about people and ways of working as it is about the tools themselves.

Claude Code in Action was particularly interesting from a practical perspective. It demonstrates how AI can support software development workflows, automate repetitive tasks, assist with code generation and review, and help teams move faster while maintaining quality. Even for those of us who are not full-time developers, it offers valuable insight into how AI is changing the way technical teams work.

The skills I learned on a this tool cover the tool usage but are more about people, skills, governance, and trusted data.

What my WorkIQ Persona Sketch taught me about Data Governance

Like most people who have spent years working in data, governance and technology, I am occasionally asked a surprisingly difficult question.

"So, what is it that you actually do?"

It sounds simple enough. After all, I spend my days helping organisations improve the way they govern data and AI. I work with executives, data leaders, governance teams and technology specialists. I write, speak, advise, mentor, and occasionally disappear down a rabbit hole researching some obscure aspect of metadata or organisational complexity. Yet whenever somebody outside the industry asks the question, the answer rarely feels satisfactory.

You can talk about Microsoft Purview, governance operating models, stewardship, responsible AI, data quality, business glossaries and metadata management. You can explain frameworks, programmes and organisational change. Most people are polite enough to listen, but somewhere around the mention of lineage or compliance controls their eyes begin to glaze over.

Recently I decided to create one of the increasingly popular WorkIQ Persona Sketches. I expected it would be an interesting visual exercise and perhaps a useful profile graphic for social media. What I did not expect was that it would force me to reflect on what my work is really about. When the first version appeared, it looked impressive enough. It showed technology platforms, governance activities, speaking engagements, research interests and community involvement. It included the various things I spend time doing and many of the topics I am associated with professionally.

The problem was that it still did not quite feel like me. It described my activities, but not my purpose. As I refined the sketch, removing some elements and emphasising others, a pattern began to emerge. The technology became smaller. The governance themes became larger. The focus shifted away from products and towards outcomes. Instead of listing tools, the sketch started telling a story.

The story was not really about Microsoft Purview, Microsoft Fabric, Power BI or SQL Server, despite all of them appearing on the page. It was about helping organisations build confidence. That may sound like a subtle distinction, but I think it matters.

Many organisations still think of governance as an exercise in control. Policies need writing, roles assigning, rules enforcing and technology deploying. Those things are certainly part of governance, but they are rarely the reason governance exists.

The organisations making the most progress with data and AI are not trying to govern for the sake of governing. They are trying to make better decisions, reduce uncertainty and trying to understand their information well enough to trust it. Increasingly, they are trying to ensure the foundations beneath their AI ambitions are solid enough to support what comes next. Governance is simply the mechanism that helps create that confidence and trust. Looking at the sketch, I realised that almost every part of my career has revolved around that idea.

My doctoral research explored the complexity of database systems and resulted in the development of the CODEX Framework. At the time, I was trying to understand how organisations could make sense of increasingly complex data environments. Years later, those same themes continue to appear in conversations about governance, AI readiness and organisational trust.

My work as a Microsoft Data Platform MVP has never really been about technology advocacy. It has been about helping people understand how technology can support better outcomes.

The conferences, community events, blogs and mentoring activities all stem from the same belief. If we help people understand data better, we help organisations make better decisions. If we improve governance, we improve confidence and the likelihood that data and AI initiatives deliver meaningful value.

The sketch made something else visible too. Many people see specialists through the lens of tools. We become the Purview person, the SQL Server person or the governance person. The reality is usually much broader. The most valuable work often happens between disciplines rather than within them. Good governance sits between business strategy and technology. It sits between risk, innovation, people, processes, culture and platforms. Success rarely comes from solving a technical challenge in isolation. It comes from helping organisations connect those pieces together in a way that makes sense.

That is why the final version of the sketch ended up with a simple statement at its centre:

Helping shape how organisations think strategically about Data & AI Governance and Microsoft Purview.

Microsoft Purview is a powerful accelerator for governance, but it is not the answer to every governance challenge on its own. Organisations still need to make governance decisions with purpose, coherence, and accountability. Governance remains the most important discipline in an organisation because it connects technology, behaviour, and business outcomes.

As I reflected on this, the sketch I was creating shifted. It became less of a personal profile and more of a map of the ideas I care about, the communities I contribute to, and the research, technology, and leadership themes that have shaped my career. Most importantly, it became a reminder that roles are easier to understand when viewed through purpose rather than activity. The common thread running through my work is the ability to connect governance, technology, and business outcomes in a way that creates integrity, clarity, and direction.

The next time somebody asks what I do, I might still mention governance, AI and Microsoft technologies.,  but I suspect I will start with something simpler. I help organisations build confidence in their data and AI so they can make better decisions. Everything else is just how that happens.

Reference 

https://github.com/pnp/copilot-prompts/tree/main/samples/prompts/m365-workiq-persona-sketch

Cabinet Level AI and How Britain’s Strategic Shift Changes the Data & AI Governance Landscape

The elevation of the Artificial Intelligence portfolio into the UK Cabinet marks a defining moment in British technology policy. With Kanishka Narayan promoted to attend Cabinet as Minister for AI, the message from Whitehall is unmistakable: artificial intelligence is no longer just a subset of digital policy or a niche driver of economic tech hubs. It is now a core pillar of national strategy, alongside economic growth, defense, and public infrastructure.

This structural shift signals that the UK intends to actively shape the global AI trajectory rather than merely adapt to it. However, accelerating AI innovation is only half the battle. Bringing dedicated ministerial oversight into the top room of government fundamentally alters how businesses, builders, and policymakers must approach data governance.

Opening the Floodgates for Innovation

For tech builders and investors, a dedicated Cabinet seat brings much needed political capital and decision-making speed. Historically, technology portfolios in government have wrestled with fragmented mandates across separate departments. Placing AI leadership directly within the Cabinet Office streamline policy across government bodies, offering clear advantages:

•  Infrastructural Investment: Delivering state of the art AI requires significant physical infrastructure from data centre capacity and grid access to supercomputing networks. Centralized ministerial authority helps unblock planning hurdles and lower energy-access barriers for compute providers.
• Public Sector Transformation: AI deployment is moving beyond private-sector start ups. Direct ministerial drive allows the government to integrate AI solutions across healthcare, transportation, and public administration, turning the state into an early anchor client for domestic innovation.
•  Global Influence: As international debates rage over technological sovereignty, safety standards, and intellectual property, having a high level AI Minister ensures Britain has a direct, unified voice in shaping cross-border regulations.

Yet, innovation does not happen in a vacuum. The speed at which a nation can deploy advanced systems is directly bounded by the strength and reliability of its data foundations.

The Heightened Need for Agile Data Governance

It is a tech adage that holds truer than ever in the generative era. An AI model is only as safe, effective, and unbiased as the data used to train and run it.

As the UK ramps up its AI ambitions, the regulatory spotlight will inevitably shine brighter on data pipelines. Rather than viewing compliance as a friction point, modern organizations must recognize governance as an essential enabler of sustainable innovation.

 1. Moving Beyond Generic Privacy Compliance

Standard GDPR compliance is no longer enough when feeding complex foundational models or automated decision engines. Organizations now face intricate queries regarding copyright, consent for machine learning uses, synthetic data generation, and systemic bias. Cabinet level prioritization will drive clearer regulatory frameworks, forcing companies to prove where their data originated and how it was processed.

2. Trust as a Competitive Differentiator

Public trust remains fragile. High profile data leaks, hallucinated outputs, or opaque automated decisions can derail enterprise initiatives overnight. Clear, transparent data governance protocols, including rigorous lineage tracking and auditability, provide the legal certainty required to deploy AI models safely at scale.

3. Fostering Regulatory Sandboxes

A centralized AI strategy enables government regulators to expand "regulatory sandboxes" controlled environments where businesses can test frontier models against real-world datasets without triggering immediate penalty risks. This gives enterprises a safe arena to experiment while establishing clear benchmarks for safety, security, and privacy compliance.

Striking the Balance: What Businesses Should Do Next

The creation of a Cabinet-level AI minister reflects a broader truth: you cannot separate the thrill of innovation from the rigor of oversight. As the UK government aligns its resources to build, attract, and scale world-leading technology, the private sector must prepare its data architecture for stricter scrutiny and faster deployment cycles.

Organizations looking to capitalize on this shift should focus on three immediate priorities

 1. Audit Data Provenance: Ensure training data and operational pipelines have clear, documented chains of ownership and consent.

 2. Implement Human-in-the-Loop Governance: Establish cross-functional AI oversight teams combining legal, engineering, and product leaders.

 3. Design for Interoperability: Build data architectures flexible enough to adapt as national standards and international compliance rules evolve.

The British government has signaled its commitment to shaping the future of AI. Now, the responsibility falls on organizations to build the trustworthy, data-driven foundations required to lead in it.

References & Further Reading

1. GOV.UK Official Announcement: Minister of State (Minister for Artificial Intelligence) Role & Profile — Official ministerial appointment details for Kanishka Narayan MP across the Cabinet Office and the Department for Business, Innovation, Science and Trade.

2. Bloomberg / The Straits Times: Burnham Picks Narayan as First British AI Minister to Attend Cabinet (July 2026) — Coverage on the elevation of the AI portfolio to Cabinet level, the restructuring of UK tech departments, and national AI infrastructure strategy.

3. ETIH EdTech Innovation Hub: Kanishka Narayan Named UK AI Minister Under Andy Burnham (July 2026) — Analysis of the UK government's strategic focus on AI innovation, industrial policy, and global competitiveness.

4. Department for Science, Innovation and Technology (DSIT): AI Safety Institute & Sovereign AI Strategy Frameworks — Policy documentation outlining UK guidelines for AI safety standards, regulatory sandboxes, and enterprise data governance.

Scaling at Cloud Speed: Moving from Manual Checklists to CDMC Automation

For years, data governance has relied on a familiar model: committees, policies, spreadsheets, and periodic reviews. It worked when data moved slowly, systems were predictable, and change could be managed through human oversight but that world no longer exists.

Today, data is created, transformed, and consumed continuously across cloud platforms. AI models are trained on that data in near real time. Decisions happen in milliseconds. And yet, in many organizations, governance is still anchored in manual controls and retrospective checks. There’s an uncomfortable truth emerging: human-in-the-loop governance cannot scale to cloud speed. The question is no longer whether governance is important. It’s whether governance can keep up and this is where the industry has been quietly converging on a new answer.

The Missing Link: Why CDMC Exists

The EDM Council didn’t create the Cloud Data Management Capabilities (CDMC) framework to replace existing governance thinking. It created it because something was missing. Frameworks like DAMA-DMBOK remain foundational they define what good governance looks like across domains such as data quality, metadata, and security. But they were never designed for an environment where:

• Data is distributed across cloud services
• Access decisions are made dynamically via APIs
• Policies must be enforced continuously not reviewed quarterly

CDMC fills that gap. It translates governance intent into 14 concrete, measurable controls, designed specifically for cloud environments, with a clear emphasis on automation and continuous enforcement. 

In other words, it moves governance from principle to execution.

From Policy to Enforcement: What Automation Really Means

The power of CDMC is not just that it defines controls, it defines controls that can be automated, monitored, and evidenced. This is a fundamental shift. Traditional governance asks: Do we have a policy? CDMC asks Is this control being executed automatically, right now, and can we prove it? Across its 14 controls spanning governance, classification, privacy, lifecycle, and architecture, CDMC embeds governance directly into the data pipeline itself. 

The impact of that shift becomes most visible when you look at a few critical controls.

Control #1: Governance Accountability in an AI World

One of the simplest, yet most powerful, requirements is this: every sensitive data asset must have a defined owner. This is not new in principle. DAMA has long emphasised stewardship and accountability but CDMC enforces it through automation ensuring that ownership fields are populated in data catalogs, monitored, and escalated when missing. In an AI-driven context, this becomes critical. If a model produces biased or incorrect outputs, the question is no longer abstract. It becomes operational:

Who owns the data that trained this model?

Without automated ownership tracking, accountability collapses. With it, organizations can trace responsibility back to the source.

Control #11: Data Privacy that doesn’t rely on Humans

Privacy has always been a governance priority. But manual processes, reviews, sign-offs, compliance checklists are no longer sufficient when data is constantly moving and being repurposed. CDMC embeds privacy into the flow of data itself. It requires automated triggers, such as data protection impact assessments for personal data, ensuring that privacy controls are activated consistently and at scale. This matters even more in AI scenarios, where training datasets can be assembled from multiple sources rapidly. You simply cannot rely on someone remembering to remove PII before it enters a pipeline. You need a system that ensures it never gets there in the first place.

Control #12: Stopping Data Swamps before they start

Data quality has always been a known challenge. What’s changed is the speed at which poor-quality data propagates. In traditional environments, issues might take weeks to surface. In AI pipelines, they surface instantly and at scale. CDMC addresses this by requiring data quality measurement as a built-in control, applied at ingestion and continuously monitored through metrics. This is a subtle but profound shift. Instead of discovering problems downstream, organizations prevent them upstream. Instead of cleaning data after the fact, they stop poor data from entering the ecosystem at all. This is how you avoid the modern equivalent of a data warehouse problem: the AI-era data swamp.

The joined-up Framework: DAMA as Constitution, CDMC as Enforcement

It’s tempting to position CDMC as a replacement for traditional frameworks but that misses the point. The real strength comes from how they work together.

• DAMA-DMBOK defines the principles of governance, the constitution that outlines what good looks like
• CDMC defines the execution, the enforcement layer that ensures those principles are actually applied

Where DAMA says:

Data must be secure.

CDMC operationalises it as:

Security controls must be enabled, monitored, and evidenced automatically for all sensitive data.

Where DAMA defines accountability, CDMC ensures accountability exists in the system. Where DAMA defines quality, CDMC ensures quality is measured continuously. This is the bridge many organizations have been missing.

From Governance Theatre to Operational Reality

There is a growing gap between organizations that talk about governance and those that have embedded it into their platforms.

Manual governance processes, however well designed, become governance theatre in cloud environments:

• Policies exist, but are not enforced
• Ownership is defined, but not maintained
• Controls are documented, but not executed

CDMC changes the conversation. It forces organisations to move from:

• Periodic assurance → continuous control
• Documentation → instrumentation
• Manual oversight → automated guardrails

And that’s what makes it so relevant in the age of AI.

AI doesn’t remove the need for governance, it increases it exponentially. But it also exposes the limits of traditional approaches. You cannot govern at cloud speed with spreadsheets, committees, and retrospective checks. You need governance that is:

• Embedded
• Automated
• Measurable
• Continuous

That’s the shift CDMC represents. Not a new theory of governance but a new way of making governance real.

References

EDM Council – CDMC 14 Key Controls & Automations

EDM Council – Cloud Data Management Capabilities (CDMC) Framework Overview

Snowflake – CDMC framework and its role in cloud governance

PR Newswire – Launch of the CDMC framework and industry adoption

Securiti – CDMC framework and implementation of automated controls

GovCDOiq – CDMC as a best-practice framework for cloud data management

Seen but Not Heard: The Age of Data Governance

There’s a phrase I remember being told as a child  “seen but not heard.”

At the time, it meant quiet compliance. Something present, something acknowledged, but not something that shaped the room or influenced what happened next. Strangely, that’s exactly how organizations have treated data governance for years. It has always been there, in the background. Policies exist, frameworks have been written, roles have been defined. If you look hard enough, every organization can point to where governance sits. It is visible. It is documented. It is technically present but it hasn’t truly been heard. It hasn’t influenced how systems are designed, how teams deliver, or how decisions are made in the way it should. Instead, governance has often been something that follows behind delivery as a correction, a control, a necessary inconvenience once the “real work” has already been done. That made sense, once but it doesn’t any longer.

What has changed is not governance itself, it is the world around it. We now operate in organizations where data is not a by-product of activity; it is the thing everything depends on. Strategy is built on it, operations are driven by it, and increasingly, decisions are delegated to systems that rely entirely on it. There is no part of a modern organization that sits outside of data anymore and yet, governance is still too often treated as if it does. That tension is becoming impossible to ignore because when every system depends on data, every issue becomes a governance issue. When numbers do not align between reports, when teams cannot agree on definitions, when ownership is unclear, when trust in outputs begins to erode these are not technical failures in isolation. They are symptoms of something deeper: a lack of embedded governance. You can see this play out repeatedly. Organizations invest in platforms, they modernise architectures, they implement analytics solutions, they adopt AI. Each initiative is presented as progress, and in isolation, it often is. But without governance woven into the fabric of these initiatives, complexity accumulates rather than resolves. Data spreads, inconsistency grows, and the ability to explain or trust what is being produced gradually diminishes. Governance, in those moments, has been seen but it was never allowed to shape the outcome.

The emergence of AI has brought this reality into sharper focus. For years, organizations could tolerate a degree of inconsistency in their data. It caused frustration, inefficiency, and occasionally risk, but it remained manageable. AI does not allow for that tolerance. It amplifies whatever it is given. Good data becomes insight at scale. Poor data becomes risk at scale. There is no neutral outcome. The old saying “garbage in, garbage out” still applies, but it now applies faster, at greater scale, and with far more impact than before. When decisions begin to be influenced or even made by systems fed on ungoverned data, the consequences are no longer contained within individual processes. They affect entire organizations. At that point, governance is no longer a supporting capability. It becomes the condition for whether anything works at all.

This is why the idea that governance can be added later no longer holds. It is not something that can sit alongside delivery or follow it. Governance determines what “good” looks like before anything is built. It defines ownership, establishes meaning, sets expectations, and ensures consistency. Without it, delivery moves forward, but coherence does not and that is the subtle but critical shift that is still being missed. We are not entering a stage where governance becomes more important as a standalone discipline. We are entering a stage where governance becomes inseparable from everything else. It is not another workstream to manage it is part of how every workstream operates. Every technology solution carries assumptions about data. Every integration defines how data flows. Every report reflects decisions about meaning, quality, and trust. Every AI model relies on choices about what data is used and how it is interpreted. In all of these cases, governance is already present. The difference is whether it has been made explicit, intentional, and embedded or whether it remains invisible until it fails.

One of the reasons organizations struggle with this shift is that governance has historically been framed in the wrong way. It has been positioned as a control mechanism, something that restricts or slows progress. It has been documented extensively, but lived infrequently. It has often been assigned to a function rather than understood as a shared organizational responsibility. As a result, it has been treated as optional in practice, even when it is mandatory in principle but when governance is embedded properly, it does not slow organizations down. It removes uncertainty. It allows decisions to be made with confidence because there is clarity around ownership, meaning, and quality. It reduces rework because expectations are clear from the outset. It enables innovation because it provides the guardrails that make experimentation safe. In other words, it makes progress sustainable.

The irony is that most organizations are already feeling the consequences of not doing this, even if they do not describe it in those terms. The questions that surface in meetings about which version of the truth to trust, about who is responsible for a dataset, about whether something can be used safely or compliantly are all governance questions. They just are not recognised as such and because they are not recognised, they are not addressed systematically. Instead, they are solved locally, temporarily, repeatedly. Governance remains visible in theory, but unheard in practice.

We are now at a point where that is no longer viable. If data is the thing that everything depends on, then governance must be the thing that everything contains. Not as an overlay, not as an afterthought, but as a standard, embedded part of how organizations operate. This is the age of data governance — not because governance is new, but because the absence of it is no longer survivable. The organizations that recognise this will not be the ones with the most advanced tools or the largest data estates. They will be the ones that understand their data well enough to trust it, control it, and use it consistently across every part of the business. They will be the ones that stop simply seeing data governance, and finally start listening to what it has been telling them all along.

The Reality of Compliance in the Age of AI

Compliance used to be retrospective. Policies were written, audits were conducted and evidence was gathered after the fact to demonstrate that controls had been followed. That approach is no longer sufficient. AI has introduced a level of complexity where decisions are made faster, data is reused in ways that are difficult to track, and accountability becomes harder to define. Compliance cannot keep up if it remains a reactive process. It has to become something that is designed into how organizations operate.

The problem beneath the surface

Most organizations still treat compliance as a separate function. A team that interprets regulation. A set of policies that sit alongside operations. A series of controls that are checked periodically but the real challenge is not understanding regulation. It is applying it consistently across processes, systems, and increasingly, AI-driven outcomes.

• What data can be used for training
• How decisions are explained
• Where sensitive information is retained or deleted

These are operational questions, not just compliance ones.

Where Purview comes in

Purview Compliance capabilities focus on managing these challenges in a structured way. Data lifecycle management defines how long data should exist and when it should be removed. Records management strengthens that by applying legal and regulatory context. Compliance Manager provides a framework to track controls and measure progress against requirements. More recently, these capabilities are being used to address AI-related concerns. Understanding data usage, managing retention, and demonstrating control are all foundational to responsible AI. The technology does not replace compliance thinking. It enables it to be applied consistently.

The technical layer that matters

Retention labels and policies are often seen as administrative tools. In reality, they directly influence how data is stored, preserved, or deleted across workloads. Records management introduces immutability and defensibility. Compliance Manager maps controls to regulatory standards, providing visibility into gaps and progress. These are not isolated features. They form a system where compliance is codified into policies that operate at scale.

Why this matters now

Regulation is evolving. The EU AI Act, data protection laws, and industry-specific requirements are all pushing organizations towards greater accountability. At the same time, AI is accelerating how data is used. This creates a tension. Organizations want to move quickly, but also need to demonstrate control. Manual processes cannot bridge that gap. Compliance has to become embedded. It has to operate continuously, not periodically.

The reality

In the age of AI, compliance is no longer about proving that controls exist. It is about proving that they are applied, monitored, and effective in a constantly changing environment. Purview provides the mechanisms to do this, but like governance and security, it depends on how it is used. Policies must reflect real business requirements. Controls must be implemented consistently. Ownership must be clear. Otherwise, compliance remains a reporting exercise rather than a capability.

References and learning

https://learn.microsoft.com/en-us/purview/compliance
https://learn.microsoft.com/en-us/purview/data-lifecycle-management-overview

What Data Leaders Must Unlearn to Lead in the Age of AI

The hardest part of leading in the AI era isn’t learning new skills, it is unlearning old assumptions. Many of the beliefs that shaped data leadership over the past decade no longer apply. The pace of change, the complexity of modern estates, and the unpredictability of AI systems demand a different mindset. Leaders must be willing to let go of outdated models of control, certainty, and hierarchy.

One of the first assumptions to unlearn is that governance slows innovation. In reality, governance accelerates innovation by reducing risk, increasing clarity, and enabling responsible experimentation. When governance is embedded rather than imposed, it becomes a catalyst rather than a constraint. Leaders who cling to the old narrative will find themselves outpaced by those who embrace governance as a strategic enabler.

Another assumption to unlearn is that documentation equals understanding. In the AI era, understanding comes from lineage, monitoring, and behavioural metadata, not static documents. Leaders must shift from documenting after the fact to embedding governance into the system itself. This requires investment in tooling, automation, and literacy.

Leaders must also unlearn the idea that AI systems can be trusted without oversight. AI is probabilistic, not deterministic. It requires continuous monitoring, not one‑time validation. The organisations that thrive will be those that treat AI as a dynamic system requiring ongoing governance, not a product that can be finished.

Finally, leaders must unlearn the belief that expertise is static. In the AI era, expertise evolves. The best leaders will be those who remain curious, adaptable, and willing to challenge their own assumptions. Unlearning is not a weakness but a leadership skill.

Fabric, Purview, and the New Shape of Enterprise Data Architecture

Fabric has reshaped the Microsoft data landscape by unifying analytics, engineering, and storage into a single experience. But unification alone does not create coherence. The real transformation happens when Fabric is paired with Purview. Together, they form an architecture where data movement, governance, and analytics operate as one system rather than disconnected components.

This convergence matters because modern data estates are too complex to govern manually. Data flows across pipelines, notebooks, semantic models, and AI workloads. Without integrated governance, organisations end up with pockets of visibility rather than a complete picture. Purview provides the lineage, classification, and policy enforcement that Fabric alone cannot deliver.

One of the most powerful aspects of this integration is the alignment between data products and governance. Fabric encourages teams to think in terms of products that are curated, reusable assets with clear ownership. Purview reinforces this by providing the metadata, stewardship, and controls that make data products trustworthy. Governance becomes part of the product lifecycle, not an afterthought.

This new architecture also supports hybrid and multi‑cloud realities. Many organisations are not all using Fabric, nor should they be. Purview’s ability to govern across environments ensures that Fabric becomes a strategic hub rather than a silo. The result is an architecture that is unified, not monolithic but flexible.

As organisations modernise their estates, the combination of Fabric and Purview will become the default pattern. It is not just a technical alignment but it is a governance first architecture for the AI era.

It is FABCON and SQLCON in Atlanta March 16 - 20, 2026. 

Why Strategic Leaders are Pivoting to Contextual Governance

For decades, data governance has been treated as a static discipline with a set of rigid policies laid out in formal frameworks and applied uniformly across the enterprise. But in an era defined by decentralized architectures and the breakneck speed of AI adoption, this one-size-fits-all approach is inefficient and has the potential to increase business risk.

The mismatch between static governance and dynamic data estates is the primary reason why many digital transformation projects stall. It is time to move toward Contextual Governance.

The Governance Friction Paradox

Traditional governance models fail because they are binary. They treat data as a fixed asset rather than a fluid utility. This creates a paradox:

• Over-governance: Smothering low-risk innovation with unnecessary red tape.

• Under-governance: Missing the subtle, high-risk nuances of how data is actually used in the wild.

Static rules rely on metadata labels that are often outdated the moment they are applied. Contextual governance, however, shifts the focus from what the data is to how the data is behaving.

What is Contextual Governance?

Contextual governance is a move from policing to orchestration. It is an adaptive framework that evaluates risk in real-time based on the intersection of three pillars:

1. The Actor: Who is accessing the data, and what is their historical behaviour?

2. The Environment: Where is the data flowing? Is it a sandboxed R&D environment or a customer-facing LLM?

3. The Intent: Is the data being used for a routine report, or is it being fed into a model that could leak proprietary logic?

The Strategic Shift: We are moving from asking, Is this data protected? to asking, Is this data protected enough for this specific moment?

Beyond Compliance: The Competitive Edge

For the C-suite and strategic leads, contextual governance isn't just a compliance checkbox. It is a performance multiplier.

• Agility at Scale: By automating the easy permissions and tightening controls risk is reduced and  the bottlenecks a removed that frustrate engineering and data science teams.

• AI Readiness: AI systems don't live in a vacuum. A model that is safe in a localized test may become dangerous when exposed to real-world edge cases. Contextual governance provides the guardrails necessary to deploy AI with confidence.

• Intelligent Foundations: This shift forces a higher standard for metadata and lineage. You are mapping data and mapping the value stream of the entire organization.

The Path Forward

Transitioning to this model requires more than new software; it requires a cultural pivot. We must change how we view governance from firm control to see it as a central intelligent system of the enterprise.

The future of data doesn't belong to those with the thickest rulebooks. It belongs to those who can govern at the speed of business.

AI Is Making Us Dumber, Part III: When Models Learn Faster Than Organisations Do

We have reached a point where models can learn faster than organisations can adapt. This creates a dangerous asymmetry: the technology evolves, but the governance, culture, and literacy lag behind. The result is a widening gap between capability and control. Organisations deploy increasingly powerful models without fully understanding their behaviour, limitations, or risks.

This gap is not caused by technology, it is caused by organisational inertia. Many teams still rely on outdated governance processes that cannot keep pace with continuous learning systems. Policies are static, reviews are infrequent, and oversight is reactive. Meanwhile, models evolve with every new dataset, every retraining cycle, and every shift in user behaviour.

The solution is not to slow the models  but it is to accelerate organisational learning. Governance must become continuous, adaptive, and embedded into operational workflows. This means real‑time monitoring, dynamic policies, and stewardship that evolves alongside the data. It also means investing in literacy so that teams understand not just how to use AI, but how to question it.

When organisations learn as fast as their models, AI becomes a strategic advantage. When they don’t, AI becomes a liability. The choice is not technological, it is cultural.

I spoke on the topic at the  Data Toboggan Winter Edition in a session entitled 'Data Literacy: The Human Advantage in an AI World.'

AI Is Making Us Dumber, Part II: When Automation Replaces Understanding

The second wave of AI dependency is more subtle than the first. It is not about hallucinations or bias, it’ is about the erosion of organisational understanding. As AI tools become more capable, teams increasingly rely on them to summarise, interpret, and decide. Over time, this creates a dangerous dynamic: people stop interrogating the underlying data and start accepting outputs at face value.

This shift is particularly risky in environments where data quality is inconsistent or poorly governed. When teams don’t understand the lineage, context, or limitations of the data feeding their models, they lose the ability to challenge results. AI becomes a black box, and decisions become detached from reality.

Governance is the antidote. By enforcing lineage, quality checks, and human‑in‑the‑loop review, organisations ensure that automation enhances rather than replaces understanding. Governance creates the conditions for informed oversight, not blind trust.

The goal is not to reduce AI usage, it is to elevate human capability alongside it. AI should accelerate insight, not diminish expertise. When governance is strong, AI becomes a partner. When governance is weak, AI becomes a crutch.

The Governance Reset: Five Data Strategy Predictions for 2026

Every January brings a wave of predictions, but 2026 feels different. The pace of change in data and AI has outstripped the pace of organisational adaptation, and leaders are beginning to recognise that their existing strategies are no longer fit for purpose. The old model of annual planning cycles, static governance frameworks, and siloed ownership simply cannot keep up with the velocity of modern data estates. This year will force a reset.

Continuous Governance

The first major shift will be toward continuous governance. Organisations can no longer rely on periodic reviews or manual controls. Governance must operate at the speed of data creation, not the speed of committee meetings. Automated lineage, dynamic classification, and policy‑driven access will become baseline expectations rather than advanced capabilities.

Clarity in areas of data Management

Second, we’ll see a rise in data contracts as a mechanism for aligning producers and consumers. Contracts bring clarity to ownership, quality expectations, and change management. They also reduce friction between teams by making responsibilities explicit. This is governance embedded into delivery, not bolted on afterward.

AI‑driven Metadata Enrichment

Third, AI‑driven metadata enrichment will become essential. Manual documentation has never scaled, and 2026 will be the year organisations finally stop pretending it can. Automated tagging, relationship inference, and behavioural metadata will fill the gaps humans never had time to address.

Cross‑functional Stewardship will Mature

Fourth, cross‑functional stewardship will mature. Governance will no longer sit with a single team; it will be distributed across product, engineering, analytics, and compliance. This shift will require cultural change, but it’s the only sustainable model.

Embrace adaptive policies

Finally, organisations will embrace adaptive policies and rules that adjust based on context, sensitivity, and risk. Static rules cannot govern dynamic estates. Adaptive governance will become the new normal.

AI is Making us Dumber Highlights the need to Fix Our Data Foundations

There’s a growing irony in the AI boom, the more we automate, the less we seem to understand. People are outsourcing judgement to models they barely comprehend, and organisations are making decisions based on outputs they can’t trace. It’s not that AI is inherently dangerous,  it is that our data foundations are often too weak to support the weight we are placing on them. When the underlying data is inconsistent, undocumented, or poorly governed, AI becomes a mirror reflecting our own gaps back at us.

The problem isn’t the technology; it’s the dependency. When teams rely on AI to summarise, interpret, or decide, they lose the ability to interrogate the underlying data. This erosion of understanding is subtle but profound. It creates a culture where speed is valued over clarity, and convenience over accountability. That’s when AI stops being a tool and starts becoming a potential issue.

The antidote is governance. Strong lineage, quality controls, and stewardship ensure that AI systems are built on solid ground. Governance doesn’t slow innovation, it stabilises it. It gives organisations the confidence to adopt AI without sacrificing transparency or control.

If we want AI to augment rather than erode our intelligence, we must invest in the foundations. AI should elevate human capability, not replace it. And that begins with knowing our data.

A couple of the ever increasing articles on the subject.

Is AI Dulling our Minds

Will AI make you stupid?

From Steam to Silicon to Sentience: Four Industrial Revolutions and the Fragile Future of AI

The story of human progress is punctuated by revolutions, not just in technology, but in how we think, organize, and trust. From the steam engines of the 1840s to the generative models of the 2020s, each wave has promised liberation and delivered disruption. Today, as AI surges toward ubiquity, we must ask: what have we learned from past revolutions, and what must we safeguard before the bubble bursts.

Four Revolutions That Changed Everything

There are four revolutions that resulted in significant change where we can learn from the affects to help the AI revolution progress unhindered.

Era	Catalyst	Impact	Risk
Industrial Revolution (c. 1760 – 1840s)	Steam power, mechanization	Mass production, urbanization, labour displacement	Exploitation, unrest (e.g. Plug Plot Riots, 1842)
Digital Revolution (1950s – 1990s)	Mainframes, UK computing pioneers, PCs	Automation, global communication, software economies	Surveillance, fragmentation, digital exclusion
Cloud Revolution (2000s – 2020s)	Virtualization, SaaS, mobile-first	Scalable infrastructure, remote work, data centralization	Vendor lock-in, opaque governance, cyber risk
AI Revolution (2020s –)	Foundation models, generative AI	Cognitive automation, new interfaces, synthetic creativity	Hallucinations, bias, job loss, trust collapse

 During the industrial revolution there was a deep industrial economic depression. The Plug Plot Riots were a wave of industrial action and disturbances across Lancashire, Cheshire, and Yorkshire, triggered by severe wage reductions (often 20-25% in the cotton and coal industries). Many workers aligned with the Chartist movement advocating for political reform, responded by "plugging" mill boilers, removing drain plugs to flood engines and halt production which forced factories to close.   The Plug Plot Riots of 1842 led to some improvements for workers, notably the prevention of further wage cuts and the eventual passage of the Factory Act 1844, which introduced limited reforms. It introduced a reduction in working hours for women and children, some safety regulations in factories and a modest step toward better labour conditions.

The second revolution of computing was not just technical. It redefined abstraction, logic, and control. From the UK’s early computing pioneers to the rise of PCs, it laid the groundwork for cloud and AI. Yet it also introduced new vulnerabilities: fragmented standards, digital inequality, and the erosion of analogue memory.

Cloud as the Bridge: Infrastructure to Intelligence

Cloud computing connected digital and AI with its abstracted hardware, centralized data, and the capabilities to scale with ease. But as Satya Nadella emphasizes in his annual letter and Microsoft’s 2025 report, innovation without strategic purpose is fragile. Microsoft’s Secure Future Initiative and Quality Excellence Initiative reflect a shift: AI must be built on trust, not just talent.

Brad Smith’s AI Diffusion Report warns that AI is spreading faster than any prior technology but unevenly. The Global South, non-English languages, and underrepresented communities’ risk being left behind.

Data: The Fuel, the Flaw, the Future

AI’s power is unprecedented and has the power to improve or destroy depending on the algorithm development but also on the state of data. Poor quality, biased, or ungoverned data leads to hallucinations, misinformation, and systemic risk. As the BBC’s article on AI hallucinations shows, even the most advanced models can confidently fabricate facts, undermining journalism, science, and public trust. From the simplest things I have seen AI fabricate data, which is written so well, to the untrained eye it could be believed. Once the data is triangulated the output can be trusted. However, the data sources quality, the prompts and data that is behind paywalls will influence the outcome.

This is not a glitch it is a consequence of probabilistic systems trained on imperfect inputs. Without rigorous data governance, provenance tracking, and human oversight, AI becomes a mirror of our worst assumptions.

When the Bubble Bursts: Coping with the AI Comedown

Every revolution has its reckoning. The Plug Plot Riots of 1842, the dot-com crash, and the decline of post-industrial towns all reveal the cost of overhyped promises and underprepared systems. When the AI bubble bursts whether through regulation, disillusionment, or economic correction, organizations with strong data foundations, ethical frameworks, and human-centred design will endure.

Those who chased novelty without governance will falter.

Satya Nadella’s mantra is “thinking in decades, executing in quarters” is more than a business strategy. It’s a survival imperative. The AI era demands long-term vision grounded in short-term accountability. That means:

- Investing in data quality and lineage as core infrastructure

- Embedding responsible AI principles into every product and process

- Preparing workers for augmentation, not just automation

- Designing for resilience, not just scale

Conclusion: From Revolution to Renaissance

The Industrial Revolution reshaped labour. The digital revolution redefined logic. The cloud revolution scaled infrastructure. AI is now rewriting cognition. but without trust, transparency, and governance, even the most powerful tools will falter. As the socio-technical divide deepens and ecological systems strain, the cost of inaction grows, and we risk accelerating collapse socially and ecologically.

The disruption from AI is only just beginning. As Business Insider quoted, “Elon Musk said AI will make desk jobs feel like when workers used to make calculations by hand before the computer age.” This echoes the upheaval of 1842, when industrialisation redefined labour.

If we want AI to be a renaissance, not a reckoning, we must treat data as infrastructure, governance as strategy, and human ethics as non-negotiable. The future isn’t just what we build; it’s what we’re willing to steward.

We must draw a line: to protect data, embed meaningful guardrails, and confront the human cost of displacement. That means planning not only for the jobs we lose, but for the ones we must invent. It also means addressing the widening continental divide in AI development and its cascading impact on the environment and global economy.

References

'It's going to be really bad': Fears over AI bubble bursting grow in Silicon Valley 

https://www.bbc.co.uk/news/articles/cz69qy760weo

Satya Nadella annual letter: Thinking in decades, executing in quarters

https://www.microsoft.com/investor/reports/ar25/index.htmlhttps://www.linkedin.com/pulse/my-annual-letter-thinking-decades-executing-quarters-satya-nadella-7orpc?utm_source=share&utm_medium=member_android&utm_campaign=share_via

Brad Smith https://aka.ms/AIDiffusionReport

Elon Musk says the AI 'supersonic tsunami' will eliminate desk jobs 'at a very rapid pace'

https://www.businessinsider.com/elon-musk-ai-supersonic-tsunami-job-displacement-future-joe-rogan-2025-11

 Transparency: Written with the help of Copilot.