Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Tuesday, 29 September 2026

OneLake is quietly becoming the trust layer for Data and AI

When Microsoft first introduced Fabric, much of the conversation focused on consolidation. The story was about bringing together data engineering, data warehousing, business intelligence and analytics into a single SaaS platform. OneLake played an important role within that narrative by providing a single storage layer that could be shared across workloads, reducing duplication and simplifying how data moved through the platform.



The announcements emerging from FabCon Barcelona suggest that Microsoft now has a much broader ambition for OneLake. While it remains the storage foundation for Fabric, many of the latest investments have surprisingly little to do with storage. Instead, they focus on governance, security, discovery, operational management and AI enablement. Viewed individually, features such as Governance Policies, Governance Insights, Domains, Secure and Catalog integration might appear to be incremental enhancements. Viewed collectively, they reveal a platform that is evolving beyond data storage and towards something much more strategic.

What stands out is that Microsoft appears to be tackling a different set of problems than those it faced when Fabric launched. Early adopters needed a platform capable of building data products and analytical solutions. Today's enterprise customers increasingly need a platform capable of operating hundreds or thousands of those assets while maintaining visibility, consistency and trust. The challenge is no longer simply creating a lakehouse, warehouse or report. The challenge is understanding who owns it, whether it can be trusted, how it should be governed and whether it can safely be used by both people and AI systems.

OneLake the operational centre of Fabric

One of the clearest signals from the latest announcements is the growing importance of the OneLake Catalog. Historically, data catalogues have been associated with discovery. They help users search for data, understand metadata and locate information assets across an organisation. What Microsoft is building increasingly goes beyond those traditional expectations.

The introduction of governance dashboards, governance policies, security administration and domain management within the catalog points towards a broader role. These capabilities are not helping users store data. They are helping organisations manage Fabric itself. The Govern experience provides visibility across the estate, Governance Policies introduce mechanisms for maintaining standards at scale, Domains establish accountability structures and the Secure experience provides oversight of access and permissions. These are operational capabilities rather than storage capabilities.

For organisations expanding their use of Fabric, this distinction matters. The technical challenge of creating analytical workloads is often relatively straightforward. The more difficult challenge is ensuring those workloads remain manageable as adoption grows. Platform teams need visibility. Governance teams need confidence. Administrators need control. The latest announcements suggest that Microsoft increasingly sees OneLake as the place where these concerns are brought together. This is perhaps one of the most significant shifts occurring within Fabric today. OneLake is no longer just where data resides. It is becoming the place where organisations understand and manage their data estate.

OneLake is becoming the discovery layer for trusted data

A second theme running through the announcements is trust. Most organisations do not have a shortage of data. If anything, they have the opposite problem. As data estates grow, users find it increasingly difficult to determine which assets are authoritative, which can be reused and which should inform business decisions. The challenge shifts from finding data to finding the right data.

Many of the catalog enhancements announced at FabCon can be viewed through this lens. Governance recommendations, endorsements, improved discovery experiences and governance insights all contribute towards helping users identify data that is trustworthy and reusable. Although these features appear distinct on the surface, they are addressing a common problem that affects almost every organisation pursuing data-driven transformation.

Trust has always been one of the more difficult aspects of governance to operationalise. Policies and standards can be documented, but users ultimately make decisions based on confidence. If they cannot easily determine whether data is owned, maintained, endorsed and understood, they will either avoid using it altogether or create their own alternatives. Neither outcome is desirable.

What makes the OneLake Catalog increasingly interesting is that it appears to be becoming the mechanism through which trust can be communicated. Ownership, endorsement, governance status, recommendations and business context are gradually being brought together into a single experience. Rather than simply helping users locate assets, the catalog is beginning to help users determine whether those assets should be used in the first place.

Consider OneLake as the Foundation for AI

The announcement that I find most revealing is not Governance Policies or the Govern experience. It is the decision to bring catalog discovery directly into Excel and Microsoft Foundry. At first glance this may appear to be a relatively modest enhancement. After all, users can already discover assets within Fabric itself. However, the significance becomes clearer when viewed from the perspective of adoption. Traditional data catalogues behave as destinations. Users must know they exist, remember to visit them and actively search for information. Microsoft appears to be pursuing a different approach. Instead of asking users to come to the catalog, the catalog is being embedded into the places where they already work.

For business users, that means trusted data can be discovered directly from Excel. For AI developers, it means information about governed organisational data can be surfaced within Foundry. In both cases, governance and discovery become part of the workflow rather than a separate activity.

This development is particularly important because AI systems face many of the same challenges as human users. An agent needs trusted information, ownership information, context and some means of understanding whether a dataset is authoritative or whether a conflicting source exists elsewhere. The role of the catalog therefore becomes increasingly important as organisations move beyond analytics and into AI-driven solutions.

Seen through this lens, the recent announcements are not simply governance enhancements. They are part of a wider effort to ensure that data can be safely discovered, understood and consumed by both people and AI systems.

The Bigger Picture

The easiest way to interpret the latest Fabric announcements is as a collection of new governance features. That explanation is not wrong, but it feels incomplete. A more interesting interpretation is that Microsoft is redefining the role of OneLake within the architecture. Governance, security, discovery and AI enablement are all being drawn closer to the storage layer and increasingly delivered through common experiences. The result is that OneLake is starting to look less like a data lake and more like the operational and trust layer that sits beneath the wider Fabric platform.

Whether this vision succeeds remains to be seen. However, the direction of travel is becoming increasingly clear. As organisations invest more heavily in Fabric, Microsoft's focus is shifting from helping customers build solutions to helping them govern, manage and trust those solutions at scale. This may not generate the same excitement as the latest AI capability, but it is precisely the sort of foundational investment that determines whether enterprise adoption can be sustained over the long term.

Reading

Build, deploy, and govern Microsoft Fabric at scale

FabCon and SQLCon 2026 in Barcelona: Building the data foundation for Microsoft Copilot and agents

Bringing governed analytics into the flow of work: Fabric Analytics at FabCon Europe 2026

What’s new across Microsoft SQL at SQLCon/FabCon Europe 2026

Microsoft Fabric at Scale: Technical Announcements for Administrators, Engineers and Platform Teams

It is the European Microsoft Fabric + SQL Community ConThe ference 28 Sep – 01 Oct 2026 in Barcelona. There are several Fabric specific announcements .  Arun Ulag’s blog FabCon and SQLCon 2026 in Barcelona: Building the data foundation for Microsoft Copilot and Agents contains a full list of announcements.




At FabCon, Microsoft's Build, Deploy and Govern Microsoft Fabric at Scale session focused less on end-user analytics and more on the practical realities of operating Fabric as an enterprise platform. The announcements were aimed squarely at the people responsible for platform engineering, DevOps, governance, security, and operational management. According to Microsoft, the themes of the session were CI/CD, agentic development, cost and resource management, observability, security, and governance, all designed to help organizations operate Fabric estates at scale without sacrificing control or compliance. Rather than discussing data products or business facing features, these announcements reflect Fabric's continued evolution into a mature enterprise platform that can support large-scale deployment, operational oversight, and governed self-service.

CI/CD Becomes a First-Class Experience. 

One of the strongest themes was Microsoft's continued investment in DevOps capabilities for Fabric.  Early Fabric adopters often found themselves balancing rapid innovation with the need for release controls, environment promotion, and deployment consistency. As Fabric workloads expanded beyond Power BI into data engineering, data science, real-time intelligence and databases, these requirements became significantly more important.

Microsoft's latest investments continue to strengthen CI/CD experiences, making it easier to treat Fabric assets as deployable platform components rather than isolated artefacts. This aligns Fabric more closely with modern software engineering practices where changes are versioned, tested and promoted through controlled deployment pipelines. For enterprise platform teams, this is particularly important because governance becomes easier when deployment processes are automated and repeatable. Manual configuration drift remains one of the most common causes of operational complexity in large data estates.

Why it matters:

Increased deployment consistency
Reduced risk of environment drift
Better support for enterprise release management
Stronger alignment between data engineering and software engineering teams

Agentic Development Arrives in Fabric

Another major direction highlighted by Microsoft was agentic development. The Fabric platform is increasingly embracing AI-assisted development experiences that help users build solutions faster while still operating within governed environments. Rather than simply generating code snippets, Microsoft's vision is for AI-powered assistants that can understand platform context, automate repetitive development tasks, and improve developer productivity.

This reflects a wider industry trend where platform engineering teams are looking to accelerate delivery without continually increasing headcount. AI tooling is becoming part of the development lifecycle itself, reducing administrative burden while helping teams navigate increasingly complex environments. The challenge for many organisations will not be whether they use AI-assisted development, but how they ensure these capabilities operate within security, compliance, and governance controls.

Why it matters:

Faster development cycles
Reduced platform administration overhead
Increased developer productivity
Better scaling of engineering teams

Observability Becomes a Critical Capability

As Fabric estates grow, monitoring becomes significantly more important. Microsoft highlighted continued investment in observability and operational visibility, helping administrators understand how capacities are performing, where resources are being consumed, and how workloads are behaving.

This is particularly relevant because many organizations have now moved beyond small pilot deployments. They are running production workloads, serving large user communities and supporting critical business processes. At that scale, platform teams need visibility into reliability, performance bottlenecks and resource consumption trends. Recent roadmap announcements further reinforce this direction with enhanced capacity monitoring, utilization insights, throttling visibility, scaling controls and capacity health indicators becoming part of the platform experience.

Why it matters:

Faster issue identification
Improved platform reliability
Better capacity planning
Reduced operational risk

Improved Cost and Capacity Management

Managing Fabric costs has become a major focus for many organisations. As adoption increases, conversations often move beyond functionality and towards sustainability. Platform owners need confidence that they can scale usage without unexpected capacity challenges or performance degradation.

Microsoft's announcements around cost and resource management indicate continued investment in helping organisations understand and control consumption. This includes better visibility into resource utilisation and more sophisticated capacity management capabilities. This is particularly important for organisations adopting a hub-and-spoke operating model where multiple business units share platform resources.

Why it matters:

Greater financial transparency
More predictable platform operations
Better chargeback and showback models
Improved utilisation of Fabric capacities

Security and Governance Continue to Mature

Security and governance were also central themes throughout the announcements. Microsoft continues to position Fabric as an enterprise-ready platform that integrates governance, compliance, auditing, security controls and administrative oversight directly into the platform. Microsoft's governance documentation highlights capabilities such as audit, workspace governance, tenant controls, monitoring, lineage, information protection and broader integration with Microsoft Purview.

The significance of these announcements is not simply that governance features exist. It is that governance is increasingly becoming operationalised as part of day-to-day platform management rather than remaining a separate compliance exercise. For platform teams, this means security and governance controls can be embedded alongside deployment, monitoring and operational management processes.

Why it matters:

Stronger enterprise compliance
Improved security oversight
Reduced governance debt
Better support for AI-ready data estates

The Bigger Picture

Taken individually, many of these announcements appear incremental. Together, however, they reveal Microsoft's strategic direction for Fabric.  Fabric is no longer just a collection of analytics workloads. Microsoft is steadily building the supporting capabilities required to operate a large-scale enterprise data platform. The focus on CI/CD, observability, cost management, security, governance and AI-assisted development demonstrates a platform increasingly designed for enterprise-scale operation rather than departmental adoption alone.  For administrators, platform engineers and governance teams, this is arguably the most important message from the session. The newest capabilities are less about creating another report or building another pipeline. They are about enabling organisations to run Fabric confidently at scale while maintaining reliability, security and control. In many ways, these were the announcements that move Fabric from a powerful analytics platform towards a mature operating platform for enterprise data, AI and analytics.

Further Reading

FabCon Fabric Governance Announcements

Microsoft's recent update, Build, Deploy and Govern Microsoft Fabric at Scale, contained several announcements focusing on a series of governance enhancements that signal a major shift in the role of OneLake within the Fabric ecosystem at FabCon. 

Historically, OneLake has been positioned as the unified data lake for Microsoft Fabric. However, the latest announcements make it clear that Microsoft's ambitions extend far beyond storage. OneLake is rapidly evolving into the place where organisations discover, govern, secure and manage their data estate at scale. This is particularly important as organisations increasingly invest in AI. Successful AI initiatives rely on trusted, well-understood and properly governed data. The announcements focus on making that trusted data easier to find, control and manage.

Announcement 1: Centralising Governance in the OneLake Catalog

The most significant announcement is Microsoft's continued expansion of the OneLake Catalog as the central hub for governance activities across Fabric. Rather than forcing administrators and data owners to move between multiple portals, governance information is being surfaced directly inside the OneLake Catalog.

The dashboard provides:

  • Estate-wide governance visibility
  • Capacity and workspace metrics
  • Domain awareness
  • Recommended governance actions
  • Links to governance tooling and administration

Data governance frequently fails due to fragmentation. Data owners work in one tool, administrators work in another, and governance teams work somewhere else entirely. Microsoft's vision is that governance becomes part of the normal Fabric experience rather than a separate activity. For organisations with growing Fabric estates, a central governance experience reduces complexity and provides a single place to understand whether data is trusted, managed and reusable.

Announcement 2: Governance Insights

Alongside the governance dashboard, Microsoft has introduced governance insights directly within the OneLake Catalog. These insights provide visibility into the overall governance state of the Fabric environment.



These insights help answer questions such as:

  • How much of our data estate is governed?
  • Which domains are most active?
  • Where are governance gaps emerging?
  • Are governance investments improving over time?

Many organisations know they have governance challenges but struggle to quantify them. Governance insights transform governance from an abstract concept into measurable information that can be reported and improved.

Announcement 3: Governance Recommendations

One of the most useful additions is Microsoft's introduction of governance recommendations. Rather than simply showing governance metrics, Fabric now highlights specific actions administrators and data owners can take to improve governance maturity. Examples include assigning domains, improving metadata quality, increasing endorsement coverage and applying sensitivity labels.

Typical examples include:

  • Increase sensitivity label coverage
  • Certify trusted data assets
  • Assign business domains
  • Improve catalog metadata

Most governance programmes excel at identifying problems. Far fewer help organisations decide where to start. Recommendations provide practical guidance that helps teams focus effort where it will have the greatest governance impact.

Announcement 4: Governance Policies

Microsoft also announced governance policies for Fabric. Policies introduce an additional layer of governance automation by allowing organisations to define governance expectations and apply them consistently across their Fabric estate. Traditionally, governance standards often exist as documents, presentations and policies that rely on users following guidance. Policies shift governance closer to platform-enforced controls.



As Fabric estates grow, organisations need governance mechanisms that scale. Policies help reduce reliance on manual reviews and improve consistency across hundreds or thousands of assets.

Announcement 5: Domain-Based Governance

Another significant development is Microsoft's continued investment in Domains. Domains allow organisations to organise data according to business functions such as Finance, Human Resources, Operations, Sales or Clinical Services. Governance information can then be analysed and reported through these business-aligned structures

One of the biggest governance challenges is accountability. Domains allow governance ownership to sit closer to the people who understand the data and use it daily. Rather than governance being entirely driven from a central team, it becomes embedded within business functions.

Announcement 6: Secure Management in the Catalog

The final major governance announcement is the introduction of a dedicated Secure experience within the OneLake Catalog. 

This provides visibility into:

  • Workspace roles
  • OneLake security roles
  • Access permissions
  • Security administration

Security and governance have traditionally been managed separately.

As organisations increasingly share data across teams and deploy AI solutions, this separation becomes increasingly difficult to maintain.

Data cannot be considered governed unless organisations can clearly explain:

  • Who owns the data
  • Who can access it
  • Why they have access

Bringing security and governance together creates a stronger foundation for trust and compliance.

Announcement 7: Bring Catalog Discovery into Excel and Microsoft Foundry

One of the more significant announcements is the expansion of OneLake Catalog beyond Fabric itself. Microsoft is embedding catalog discovery directly into products such as Excel and Microsoft Foundry. Rather than expecting users to open Fabric to search for trusted data, data discovery becomes available within the tools where people are already working. 

Data catalogues have existed for years, but they often suffer from a common problem. People only use them when they deliberately go looking for them. Many business users spend most of their day in Excel. Increasingly, AI developers and data scientists are spending time in Microsoft Foundry building AI solutions. These users may never open Fabric directly, yet they still need access to trusted and governed data. By bringing catalog discovery into these experiences, Microsoft is reducing the distance between users and governed data.

This announcement addresses one of the biggest challenges in governance: adoption. A data catalogue only creates value when people use it. Embedding catalog discovery into familiar tools increases the likelihood that users will:

  • Reuse existing trusted assets.
  • Find certified data products.
  • Understand ownership.
  • Discover business context.
  • Avoid creating duplicate datasets.

For organisations investing in AI, the implications are even more significant. AI developers using Microsoft Foundry can discover trusted organisational data directly from their development environment rather than relying on tribal knowledge or manually maintained data inventories. In simple terms, Microsoft is moving governance closer to the point of consumption.

What This Means for OneLake

When viewed together, these announcements reveal Microsoft's broader strategy. OneLake is no longer simply the storage layer for Fabric. It is becoming the platform through which organisations:

  • Discover data
  • Understand data
  • Govern data
  • Secure data
  • Share data
  • Prepare data for AI

For organisations pursuing AI initiatives, this evolution is particularly important. AI systems need trusted data sources, clear ownership, strong security controls and confidence in data quality. The latest Fabric announcements move OneLake significantly closer to becoming the operational foundation for that trusted data estate. The message from Microsoft is seems is to use OneLake to discover, govern, secure and trust your data at enterprise scale.

Friday, 25 September 2026

Big Data London emerging insights

It was great to be at BigDataLDN this week and see the changing landscape. 40% of all sessions had AI in the title. This is a significant shift compared to pre 2023 programmes, where AI titled sessions were typically less than 15%.

The 2026 landscape highlighted a number of shifts.

AI is no longer a track, it is the spine of the conference embedded across every discipline.

Governance and trust are now centre stage. Multiple sessions explicitly focused on trusted AI foundations, AI governance maturity, scaling AI safely, data governance as the prerequisite for AI and observability for AI systems.

Agentic AI is emerging as a major theme. This is the first year agentic AI had a dedicated theatre signalling a shift from the previous experimentation phase to operationalisation.

AI plus organisational transformation is the new battleground asking questions like how do we scale AI safely,  modernise legacy environments,  demonstrate measurable business value, and improve governance without slowing innovation.  This is a change from asking which model to use towards to how do we run an AI enabled organisation. 

Data governance is finally being treated as strategic and it was the strongest governance representation BigDataLDN has ever had.

AI for societal impact is gaining traction with talks now covering climate change and humanitarian impact broadening AI use beyond commercial use cases.

The keynotes were AI centric which created a conference theme of AI first.

I think the BigDataLDN conference has quietly repositioned itself an AI governance, AI strategy, AI engineering and AI transformation conference. 

AI governance seems like the new cloud migration in that it requires a change of mindset. The agenda showed a pattern where
  • AI adoption is assumed. 
  • AI scaling is the challenge. 
  • Governance is the bottleneck. 
  • Data foundations are the dependency. 
  • Operating models are the differentiator

The conference was packed with people and exhibitors and many people spent their time queueing to get into rooms to get to listen to talks.
 

Thursday, 17 September 2026

Responsible AI in 2026: Governance Moves from Principle to Practice

Microsoft's latest article, Responsible AI in 2026: How We Are Adapting for What's Ahead, highlights something many of us working in governance have been seeing for some time: AI governance is no longer a future concern. It is becoming an operational necessity. 

As AI capabilities continue to accelerate, particularly with the rise of agentic AI, the governance challenge is changing. Traditional governance approaches were largely focused on data, systems, and applications. Increasingly, organisations must also govern autonomous actions, agent interactions, tool permissions, and dynamic decision-making processes. Microsoft describes this as a move towards more adaptive governance, where controls evolve alongside the capabilities and risks of AI systems. 






















What I found most interesting is that the article places relatively little emphasis on the models themselves and much more emphasis on governance, risk management, monitoring, and assurance. Microsoft explicitly states that model capability alone will not determine AI's impact. Success will depend on whether organisations can govern AI with the rigour and adaptability needed to earn trust. 

This mirrors a trend I am seeing across the market. Many organisations are still focused on AI adoption, Copilot deployments, and proof-of-concepts. However, the harder question is emerging quickly: how do we maintain visibility, accountability, and control once AI becomes embedded in day-to-day operations?

The answer is unlikely to be found in technology alone. Microsoft's report discusses governance frameworks, risk management processes, evaluation capabilities, training, standards, and industry collaboration. These are all governance disciplines rather than purely technical controls. 

For data governance professionals, this should sound familiar. The foundations that organisations have spent years developing around ownership, accountability, quality, security, and compliance are becoming even more important in an AI-enabled world. AI governance is not replacing data governance. It is extending it.

Perhaps the most significant message from the article is that responsible AI cannot be treated as a static policy document. Microsoft describes governance as a continuous lifecycle activity that must evolve as systems learn, interact, and operate in increasingly complex environments.  That is a valuable lesson for every organisation currently exploring AI. The conversation is no longer about whether governance matters. The conversation is about whether governance can keep pace with AI's rapid evolution.

As Microsoft's latest transparency report demonstrates, the organisations most likely to succeed with AI will not simply be those with access to the best technology. They will be the organisations that can combine innovation with trust, control, and effective governance.

References

https://blogs.microsoft.com/on-the-issues/2026/09/01/responsible-ai-in-2026-how-we-are-adapting-for-whats-ahead/

Tuesday, 15 September 2026

Governance must keep pace with AI and be embedded in every stage

Over the last few weeks, the conversation around artificial intelligence has taken an increasingly dramatic turn. Following Dario Amodei's essay, We Must Pace the Frontier, and widespread media coverage of warnings from researchers and technology leaders, discussions about AI have become dominated by questions of existential risk, cyber warfare, loss of control and the possibility that advanced systems could outpace human oversight. Amodei's central argument is that the rate of AI capability development may be accelerating faster than our ability to understand, govern and safely manage those capabilities, creating a situation where precaution needs to catch up with progress.

These AI fears made me think of pushing beyond design limits where Donald Campbell’s final attempt in 1967 on Coniston Water pushed Bluebird K7 past 300 mph far beyond its original design rating of 250 mph. This pushing technological boundaries to shatter another world record, demonstrated that accelerating past design limits without evolving the safety framework exposes fatal vulnerabilities.
















While these concerns deserve serious consideration, I have been struck by how many of the proposed solutions focus on slowing AI itself. The assumption seems to be that if technology advances too quickly, the safest response is to reduce the speed of innovation until regulators, policymakers and society have time to react. However, I am not convinced that slowing AI addresses the underlying issue. The problem is not that artificial intelligence exists or that organisations are finding new ways to apply it. The problem is that governance continues to lag behind technological change, despite decades of evidence showing that this always creates unnecessary risk.

Every major technological shift follows a remarkably similar pattern. Organisations become excited by new capabilities, investment accelerates, adoption grows rapidly and governance is treated as something that can be addressed later. Eventually the consequences of that approach become visible, whether through security incidents, compliance failures, poorly understood risks or loss of trust. The discussion then turns towards regulation, controls and accountability. What is often forgotten is that governance could have been embedded from the beginning.

The current debate around AI increasingly focuses on the possibility that advanced systems may one day become difficult to control. Yet many organisations are already struggling with far more immediate challenges. They do not know who owns critical datasets. They cannot consistently identify authoritative information. They have limited visibility of the quality of the data entering analytical platforms. They have duplicated reports, conflicting definitions and inconsistent security controls. These are not theoretical future concerns. They are today's governance problems, and AI simply amplifies them.

This is one of the reasons I find the current distinction between data governance and AI governance increasingly key. AI governance is undoubtedly important, particularly as organisations begin deploying copilots, autonomous agents and decision-support systems. However, the majority of the risks associated with AI are ultimately rooted in issues that data governance has been trying to solve for years. Questions about ownership, accountability, transparency, lineage, quality, security and trust do not suddenly appear because an organisation deploys an AI model. Those questions already existed. AI merely exposes them more quickly and at greater scale.

Consider the current wave of Microsoft Copilot deployments taking place across both public and private sector organisations. There is understandable excitement about productivity gains and new ways of working, but Copilot does not create knowledge. It surfaces what already exists inside the organisation. The challenge is the state of the information environment that AI is consuming.

What concerns me most is that governance is still frequently discussed as if it were a specialist discipline owned by a single team. The reality is that the next generation of technology will make that approach increasingly difficult to sustain. As organisations move towards more autonomous forms of AI, governance decisions will need to be incorporated directly into project delivery, operational processes, architecture reviews, software development lifecycles and technology investment decisions. It will not be sufficient to maintain a separate governance workstream running alongside change initiatives. Governance will need to become a fundamental characteristic of how change is delivered.

This becomes particularly important when considering the rise of agentic AI. Much of today's governance discussion focuses on whether an AI model is accurate, fair or explainable. Those questions remain important, but autonomous systems introduce an entirely new set of concerns. Organisations will need to understand who is accountable for actions taken by an agent, what permissions it possesses, how its behaviour is monitored, when human intervention is required and how decisions are audited. These challenges cannot be resolved through model governance alone. They require broader governance frameworks that connect business ownership, risk management, security and information management.

For this reason, I believe the debate about whether we should slow AI down is asking the wrong question. The more important question is whether governance can evolve quickly enough to keep pace with innovation. History suggests that organisations are capable of managing significant technological change when appropriate governance structures are embedded from the outset. We have done this with financial controls, health and safety, privacy, cyber security and regulatory compliance. None of these disciplines emerged because organisations stopped innovating. They emerged because innovation required new forms of oversight and accountability.

If the concerns raised by Dario Amodei prove justified, then the answer is unlikely to be found solely through reducing the pace of technological development. The more sustainable response is to ensure that governance develops at the same speed as the technologies it is intended to support. Data governance, AI governance, security governance and risk management should not be viewed as separate initiatives competing with innovation. They should be recognised as the mechanisms that make innovation sustainable.

The future of AI will undoubtedly introduce challenges that we have not yet anticipated. However, organisations do not need to wait for hypothetical existential threats before they strengthen governance. The foundations are already well understood. Ownership, accountability, transparency, stewardship, good data quality, security and trust remain as relevant today as they were before the first large language model entered the public consciousness. The difference is that AI has transformed these disciplines from desirable good practice into essential business capabilities.

The organisations that succeed over the next decade will not necessarily be those that adopt AI first or deploy the greatest number of models. They will be the organisations that recognise governance as an enabler of innovation rather than a constraint upon it. In a world where AI is becoming embedded into every platform, every process and every decision, governance must become equally pervasive. The challenge is not slowing AI down. The challenge is ensuring that governance finally catches up.

References

We Must Pace the Frontier https://darioamodei.com/post/we-must-pace-the-frontier

The Guardian — “‘We must slow the pace’: CEO of Anthropic calls for an AI slowdown

https://www.theguardian.com/technology/2026/sep/12/we-must-slow-the-pace-ceo-of-anthropic-calls-for-an-ai-slowdown

TechRepublic — “Altman, Musk Back Amodei’s AI Warning: The Frontier May Be Moving Too Fast” https://www.techrepublic.com/article/news-amodei-altman-musk-slow-frontier-ai/

BBC Why are there concerns AI could threaten humanity, and how real are they? https://www.bbc.co.uk/news/articles/c790xvnzgnno

BBC AI 'kill switch' may need to be mandatory, Anthropic co-founder tells BBC https://www.bbc.co.uk/news/articles/cqgk5e2j0gg8o

BBC Anthropic researcher believes more than 10% chance AI 'could kill all humans' https://www.bbc.co.uk/news/articles/ckgwy1k42w4o

Microsoft Fabric Ontology: The missing layer between Data Governance and AI

The data industry has spent the last twenty years focused on one primary challenge: connecting data. We built and created increasingly sophisticated ways of moving information between systems and making it available for analytics. Many organisations today still struggle with a much simpler problem and that is clarity on terminology. Different department terms often mean different things when they use the same business terms.

A customer means one thing in CRM, another in finance and something slightly different again in marketing. Product definitions vary between commercial teams and operational systems. Employee records, supplier information, and assets frequently exist across multiple applications, each with its own interpretation and business rules.

Humans have become relatively adept at navigating these differences because they understand organisational context. AI does not understand this. As organisations increasingly adopt Copilot, AI agents and intelligent business applications, the challenge is no longer giving AI access to data but giving AI the meaning of terms.

This is where Microsoft Fabric Ontology, currently in preview as part of Fabric IQ, becomes particularly interesting. Microsoft describes Ontology as a machine understandable representation of enterprise vocabulary that defines business concepts through entity types, properties and relationships, creating a shared business context layer that can be used across teams, applications and AI agents. 

What makes this significant is not the technology itself but how important the need is for AI to have business understanding, just as much as it requires data access.

The problem we have been trying to solve for years

Anyone who has worked in data governance will recognise this challenge immediately. We have built business glossaries, data dictionaries, conceptual models and reference architectures in an attempt to create consistency across the organisation. Governance programmes have invested significant effort defining critical data elements, agreeing business terminology and establishing ownership for key information assets. The difficulty has always been turning those definitions into something operational. Many governance initiatives successfully define what a customer is, but those definitions often remain trapped in documents, spreadsheets or governance tools that sit separate from the systems actually using the data. The glossary becomes a reference point for people rather than an active component of the architecture. As a result, governance knowledge frequently exists in one location whilst operational data exists somewhere else. Both are valuable, but the connection between them is often weak.

Microsoft's vision for Ontology appears to be closing that gap. Rather than maintaining business definitions separately from data, Ontology allows organisations to define core business concepts and then bind those concepts directly to data residing within OneLake, Power BI semantic models, lakehouses and other Fabric data sources. The result is that the business definition and the physical data become connected through a common semantic layer. From a governance perspective, the business glossary stops being passive documentation and becomes a part of how information is understood and consumed throughout the platform.

Why AI changes everything

Inconsistencies in business terminology are often frustrating but manageable. Analysts learn the system nuances and data engineers write transformation logic to reconcile differences. Often reporting teams spend their time explaining why the numbers vary between departments.

AI fundamentally changes the scale of the problem. When an AI agent is asked a question such as Which customers are most at risk of churn? it needs more than access to customer records. It needs to understand what a customer is, which systems contain authoritative information, how related concepts connect to one another and which business rules should be applied during analysis. Without that context, even a highly capable model can produce inconsistent or misleading outcomes. 

Microsoft specifically highlights Ontology as a shared business context layer that can be consumed by Fabric agents and AI-driven workflows to support reasoning and actions across domains. Rather than asking questions against individual tables, users and AI agents can query business concepts that already carry organisational meaning.

As organisations move beyond simple AI assistants and towards agentic architectures where AI systems are expected to make decisions, execute processes and reason across multiple business domains having this tool is important. The quality of decisions will depend heavily on the quality of the organisational context provided to them.

More than another Semantic Model

Ontology is not simply another version of a semantic model. Semantic models primarily exist to simplify analytics and reporting. They provide a business friendly representation of data designed to support measures, calculations and reporting experiences.

Ontology aims to tackle a much broader challenge. It introduces concepts such as entity types, properties and relationships that represent how the organisation understands the world. Customer, Supplier, Product, Contract and Asset become business entities that exist independently of any particular source system. Relationships become explicit rather than buried inside data models and joins.

Microsoft also introduces a graph representation that allows relationships between entities to be stored and queried directly. In practical terms, this means understanding not just what something is, but how it connects to everything around it. Customers place orders. Suppliers provide products. Employees manage projects. Assets support services. These connections become part of the semantic model itself rather than logic recreated repeatedly by individual development teams. This kind of contextual understanding  for AI is enormously valuable because reasoning is often driven by relationships as much as by data values.

Is Ontology replacing Microsoft Purview?

One of the most common questions I have seen since the announcement is whether Ontology makes Microsoft Purview less relevant. Ontology and Purview address different layers of the same challenge.

Purview focuses primarily on understanding, governing and protecting information assets. It discovers data, provides lineage, manages classifications, supports compliance activities and enables organisations to establish trust in their information landscape.

Ontology focuses on meaning. 

Where Purview helps answer questions such as Where is this data?, Who owns it?, How sensitive is it? and Where did it come from?, Ontology helps answer questions such as What does this represent?, How does it relate to other business concepts? and How should AI reason about it?

The two capabilities compared.

CapabilityMicrosoft Fabric OntologyMicrosoft Purview
Primary objectiveCreate shared business meaningGovern and manage enterprise data
Key focusBusiness concepts and relationshipsData assets and metadata
Business glossaryOperational semantic layerGovernance glossary and terminology
AI supportGrounding and business reasoningTrusted metadata and governance controls
RelationshipsBusiness relationships between entitiesData lineage and technical relationships
Graph capabilitiesNative graph-based business contextMetadata and lineage visualisation
Data discoveryBound Fabric data sourcesEnterprise-wide discovery
ClassificationLimited focusCore capability
ComplianceNot a primary objectiveCore governance capability
Security and riskRelies on platform controlsGovernance, risk and compliance controls
Typical audienceAI teams, business architects, domain expertsData governance, security and compliance teams

What this means for the Future of Governance

For me, the most significant aspect of Ontology is what it says about the future direction of governance. Historically, data governance was largely created for people. Policies were written for humans, glossaries were maintained for humans. with data standards interpreted by humans. Increasingly, we need governance artefacts that machines can understand directly. AI agents, Copilots and autonomous systems cannot read a governance policy and infer organisational meaning in the way people do. They need structured, machine readable context. They need agreed definitions and relationships. Also business vocabulary they can reason over consistently is required.

Ontology appears to be Microsoft's recognition that the next generation of governance must serve both humans and machines. As AI becomes embedded within business operations, organisations will increasingly discover that trusted data is only part of the equation. Equally important is ensuring that AI understands what that data actually means. Data governance professionals have argued for years that data without context has limited value. In the era of enterprise AI, that statement feels more true than ever. Trusted AI requires trusted data, but it also requires trusted meaning. Fabric Ontology is a significant step towards delivering that meaning at scale.

Reference

What is ontology (preview)?

Microsoft Tools for Making Data AI-Ready


Building AI for Human Flourishing: Inside Microsoft’s Humanist AI Code of Conduct

As AI accelerates toward super intelligent capabilities, Microsoft has taken a bold and transparent step: publishing the draft Humanist AI Code of Conduct today, a governing framework designed to ensure future AI systems remain safe, aligned, and firmly under human control.



At its core, the Code of Conduct is built on a simple premise people matter more than AI. Technology should amplify human wellbeing, expand opportunity, and strengthen human judgment rather than replace it. Microsoft’s Humanist AI approach rejects the pursuit of AI personhood or consciousness like behaviour, instead focusing on systems that are powerful, purposeful, and safely constrained.

The document outlines several pillars:

  • Human Control & Safety — AI must remain subordinate to humanity, never resisting shutdown, redirection, or oversight. Safety constraints cannot be overridden by users or operators.  
  • Clear Boundaries — Models will not assist with weapons, mass harm, offensive cyberattacks, manipulation at scale, or any content that violates human dignity or child safety.  
  • Human Flourishing — AI should enhance learning, creativity, collaboration, and wellbeing, helping people make better decisions without replacing personal growth or human relationships.  
  • Pluralism & Inclusion — AI should support diverse cultures, values, and perspectives while upholding universal human rights and avoiding harmful bias.  
  • Transparency & Public Input — Microsoft is inviting global feedback to refine the Code before it becomes the governing blueprint for model development in 2027 and beyond.

This draft marks a significant moment: a major technology company openly sharing how it intends to build and govern superintelligent systems. It signals a future where AI is not an autonomous force but a carefully designed partner, shaped by human values, guided by public dialogue, and constrained by safety-first engineering.

References 
Humanist AI Code of Conduct
https://microsoft.ai/code-of-conduct/

Sunday, 13 September 2026

Lineage aware AI in Microsoft Fabric: A New Era of Intelligent Data Context

Microsoft Fabric has introduced a preview feature that quietly unlocks something powerful: programmable lineage. With the new Item Relations API, developers and data teams can finally access the same dependency information shown in the Fabric lineage view but now through REST endpoints that automation and AI can understand.
Lineage has always been essential for responsible data engineering. It tells you how items connect, what depends on what, and where changes might cause disruption. Until now, that insight lived mostly inside the Fabric UI. The new API changes that completely.

What the API enables
The Item Relations API exposes upstream and downstream relationships for any Fabric item. It also returns typed edges such as shortcuts, associations, orchestration links, and push‑data flows giving a structured map of how the data estate fits together.

This means the tools can now:

- Perform impact checks before modifying or deleting assets  
- Generate automatic documentation that stays current  
- Build governance dashboards driven by real dependency graphs  
- Give AI agents the context they need to reason about data safely  

Instead of guessing, the automation can now see the actual lineage.

Why this matters for AI
AI systems are only as smart as the context they receive. By exposing lineage through an API, Fabric allows AI to understand not just the content of a dataset, but its role in the wider environment. That’s a foundational step toward safer, more reliable AI‑driven automation.

A strategic preview
Although still in preview, the Item Relations API signals a shift: Fabric is turning metadata into an actionable surface for engineering, governance, and AI. It’s a small feature with big implications for anyone building responsibly in a complex data landscape.

Reference
Lineage-aware AI with the Fabric item relations API (Preview)

Tuesday, 8 September 2026

The Hierarchy of AI Oversight

Organisations frequently treat Responsible AI, AI Governance, and Data Governance as synonymous concepts. In practice, they represent three distinct, interdependent structural tiers. Treating them as interchangeable obscures how AI systems are built, verified, and operationalised within an enterprise.

A useful way to conceptualise this structure is through a three-part stack:

  •  Responsible AI defines organizational intent and boundaries.
  •  AI Governance establishes operational execution and control mechanisms.
  •  Data Governance manages the underlying assets and pipeline inputs.
When any single tier is neglected, the entire oversight framework becomes ineffective.

Responsible AI: Establishing Strategic Intent

Responsible AI sits at the top of the stack as an explicit declaration of intent. It articulates an organisation's risk tolerance, core values, and societal commitments regarding automated systems.
This layer does not detail specific technical configurations or workflow steps. Instead, it defines the overarching ethical perimeter, addressing core themes such as non-discrimination, explainability, safety, and accountability.

Key questions addressed at this layer include:
  •  What operational boundaries define acceptable versus unacceptable AI deployments?
  •  What specific harms must system designs actively prevent?
  •  What baseline commitments are required for external stakeholders and regulatory bodies?
While Responsible AI acts as the strategic compass, policy statements alone do not alter system behavior. Without operational enforcement, policy declarations remain purely symbolic. Operationalising these policies requires the secondary layer: AI Governance.

AI Governance: Implementing Operational Control

AI Governance provides the operational apparatus required to enforce Responsible AI policies. It consists of the decision rights, verification protocols, audit trails, and risk taxonomies that manage an AI model across its complete lifecycle.

This tier shifts abstract commitments into concrete engineering and management workflows. It covers model validation standards, change management, automated drift detection, and post-deployment monitoring. Systems like the GRAICE framework operate within this domain to standardise evaluation criteria.

Key questions addressed at this layer include:
  • Which roles hold approval authority at distinct stages of model development?
  • What quantitative evidence is required prior to production deployment?
  • How are performance degradation, bias drift, and unexpected edge cases detected and remediated?
  • What specific conditions trigger a mandatory model recall or pause?
AI Governance ensures that models operate within defined parameters over time. However, governance controls cannot ensure model integrity if the underlying inputs are flawed. Control frameworks require verifiable data inputs, which depends entirely on the foundational layer.

Data Governance: Securing the System Inputs

Data Governance manages the quality, legal basis, security, and lineage of the data fed into machine learning pipelines. Because statistical models reflect the characteristics of their training data, AI performance is constrained by the quality of its underlying data architecture.

Without robust data management, model output becomes inherently unpredictable. Issues such as unverified data sources, unrecorded pipeline transformations, or demographic skew directly compromise model outputs regardless of how stringent the AI control checks are.

Key questions addressed at this layer include:
  •  What is the precise lineage and chain of custody for training and validation datasets?
  •  Do clear usage rights, legal bases, and consent frameworks exist for the ingested data?
  •  Is the dataset representative, accurate, and properly versioned?
  •  How are data access controls and privacy-preserving techniques maintained through the pipeline?
Strong Data Governance provides the verifiable evidence base that AI Governance relies on. Without it, validation processes lack technical substance
.
Structural Pitfalls of Top-Down Implementation

A common failure mode occurs when organisations implement oversight from the top down. Leadership teams often publish high-level ethical guidelines and establish oversight committees before building the necessary operational controls or securing data infrastructure.

This top-down approach creates several operational vulnerabilities:
  •  Oversight committees evaluate systems without reliable technical lineage or performance data.
  •  Data quality defects and unverified assumptions are identified late in production rather than during ingestion.
  •  Ambiguity surrounds technical accountability when failures occur.
  • Defining ethical principles without establishing underlying governance frameworks leads to superficial compliance—where policy commitments exist on paper but cannot be verified or enforced at the engineering level.
Building a Cohesive Oversight Framework

Establishing an effective oversight framework requires starting from foundational technical controls and building upwards:
  • Establish Data Integrity: Secure data lineage, document legal rights, enforce validation checks, and maintain clear data stewardship across all pipelines.
  • Deploy Control Architectures: Implement repeatable stage-gate approvals, continuous testing protocols, risk logging, and lifecycle monitoring.
  • Align Operational Controls with Policy Boundaries: Connect technical metrics and threshold alerts directly to high-level organizational principles and regulatory requirements.
Aligning these three disciplines transforms AI oversight from a collection of isolated policies into an integrated operational capability.

Thursday, 3 September 2026

Governance Capabilities for High-Risk AI in the EU AI Act

Much of the discussion around the EU AI Act focuses on obligations, classifications, and compliance deadlines. While those are important, they can also obscure a more interesting point. The Act is not simply creating another regulatory checklist. It is describing the governance capabilities organisations need if they want to develop, deploy, and operate AI safely and responsibly at scale.




This becomes particularly clear when looking at Articles 8-15. Rather than a collection of disconnected requirements, these articles describe a connected operating model. They bring together governance, risk management, data quality, transparency, human oversight, documentation, and security into a framework that supports trustworthy AI throughout its lifecycle.

The infographic accompanying this article visualises those capabilities as a connected system rather than a sequence of isolated controls. Before exploring each capability, it is worth understanding where Articles 8-15 sit within the broader structure of the AI Act.

The Risk-Based Foundation of the EU AI Act

The EU AI Act adopts a risk-based approach to regulation. Rather than treating every AI system equally, it classifies systems according to the level of risk they present.

At the top of the pyramid are applications considered to represent an unacceptable risk. These uses are prohibited because they are considered incompatible with European values and fundamental rights. Below this sit High-Risk AI Systems, which are subject to the most extensive governance requirements. Beneath these are Limited Risk and Minimal Risk systems, where obligations are significantly lighter.

This distinction is important because Articles 8-15 are primarily concerned with the governance capabilities required for High-Risk AI Systems. They define what organisations must have in place to demonstrate that these systems are designed, operated, and monitored appropriately.

Governance and Accountability

Effective AI governance starts with accountability.

Although Article 8 focuses on compliance with the requirements applicable to high-risk systems, this is closely linked to the Quality Management System requirements described later in Article 17. Together, they establish the expectation that organisations must have clear governance structures, defined responsibilities, documented processes, and mechanisms for continuous improvement.

This is often where governance discussions become overly procedural. In practice, what matters is whether accountability exists. Who owns decisions? Who approves risk acceptance? Who monitors outcomes? Who intervenes when issues arise?

Organisations that treat governance as a collection of policies frequently struggle to answer these questions. Those that build governance into their operating model tend to have far greater confidence in how AI is being used and controlled.

Risk Management

One of the most significant requirements within the AI Act is the expectation that risk management is continuous.

Article 9 requires organisations to establish, implement, document, and maintain a risk management system throughout the entire lifecycle of a high-risk AI system. This is not a one-off assessment performed during development. Risks must be identified, evaluated, mitigated, monitored, and reassessed over time.

This reflects a broader reality of AI. Models evolve, data changes, user behaviour shifts, and operating environments become more complex. The risks associated with an AI system today may not be identical to those that emerge six months from now.

A mature governance programme therefore treats risk management as an ongoing capability rather than a project activity.

Data Governance

No governance framework can compensate for poor-quality data.

Article 10 recognises this by placing significant emphasis on the quality and governance of training, validation, and testing datasets. Organisations must consider data provenance, representativeness, relevance, completeness, and bias mitigation.

Many AI governance conversations focus heavily on models while paying less attention to the data that underpins them. Yet data remains one of the strongest determinants of whether an AI system will behave as intended.

This requirement is also one of the clearest areas where tools such as Microsoft Purview can support governance objectives. Data lineage, metadata management, business glossaries, and data quality capabilities provide organisations with the visibility needed to understand where data originates, how it moves, and whether it can be trusted for AI use cases.

Data governance is not a separate discipline sitting alongside AI governance. It is one of its foundational components.

Documentation and Evidence

Good governance depends upon evidence.

Articles 11 and 12 establish the requirements for technical documentation and record keeping. Organisations must maintain sufficient documentation to demonstrate conformity with regulatory obligations and provide evidence regarding how the system operates.

Technical documentation includes information such as system design, intended purpose, performance characteristics, testing activities, and risk assessments. Record keeping focuses on logs, traceability, and the ability to reconstruct events when needed.

This may appear administrative at first glance, but it plays a critical role in building accountability. When questions arise about an AI system's behaviour, organisations need more than assumptions or recollections. They need evidence. Documentation transforms governance from intention into demonstration.

Transparency and Explainability

A system cannot be governed effectively if nobody understands how it should be used.

Article 13 requires high-risk AI systems to be sufficiently transparent so that deployers can interpret outputs and use the system appropriately. Users must be provided with information about intended use, limitations, and operational considerations.

Transparency is often reduced to explainability discussions, but it extends beyond technical explanations of model behaviour. It also encompasses user guidance, operational context, and clarity regarding what the system should and should not be used for.

Many governance failures occur not because the AI was technically flawed but because people misunderstood its outputs or relied upon it in inappropriate ways. Transparency helps prevent those misunderstandings.

Human Oversight

One of the most important themes within the AI Act is the continuing role of human judgement.

Article 14 requires organisations to design systems that enable appropriate human oversight. This includes mechanisms for review, escalation, intervention, and, where necessary, stopping or overriding the system.

The phrase "human in the loop" is often used when discussing AI oversight, but the Act's expectations are broader than that. Effective oversight requires authority, competence, and accountability, not merely human presence. People need to be able to challenge outcomes, recognise anomalies, and take action when circumstances demand it. Governance remains a human responsibility, even when decisions are increasingly supported by AI.

Accuracy, Robustness and Security

The final capability area focuses on operational resilience.

Article 15 requires high-risk AI systems to achieve appropriate levels of accuracy, robustness, and cybersecurity throughout their operational life. Organisations must consider not only normal operating conditions but also errors, failures, misuse, and malicious attacks.

This reflects an important shift in thinking. Governance is not solely about policies and controls. It is also about operational performance.

An AI system that cannot remain reliable, secure, and resilient under real-world conditions cannot ultimately be considered trustworthy.

Governance is more than Compliance

When viewed together, Articles 8-15 reveal something that is often missed in discussions about the EU AI Act. The regulation is not describing a set of independent controls. It is describing a connected governance system.

Risk management relies on trustworthy data. Transparency depends on documentation. Oversight requires accountability. Security depends upon effective governance. Each capability supports the others.

This is why organisations should resist the temptation to approach the AI Act as a compliance exercise alone. The most successful governance programmes will be those that use these requirements to establish sustainable operating models that support responsible AI adoption at scale.

Ultimately, the organisations that thrive in the AI era are unlikely to be those with the longest policy documents. They will be those that can demonstrate consistent, repeatable, and accountable governance across their AI estate. That is the broader message embedded within Articles 8-15, and it is arguably far more significant than compliance alone.