Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Wednesday, 18 November 2026

Purview and OneLake Catalog Transition

 



Microsoft Ignite 2026: AI is moving from answers to action

I watched the Microsoft Ignite 2026 keynote expecting the usual mixture of product announcements, demonstrations and new terminology.

Microsoft framed the keynote around human ambition, intelligence and trust, describing them as architectural principles rather than aspirations. That distinction is important. Microsoft is building a stack around keeping human ambition at the centre, allowing organisations to retain control of their intelligence and learning, and creating the trust needed for AI to operate at scale.

Satya's summary was that we are entering a period of ubiquitous intelligence. AI will not sit in one application or one Copilot. It will increasingly be embedded in our work, data, applications, infrastructure and security. He described Copilot as “the operating system for work”, bringing together Chat, Cowork, Code and Autopilot within a common experience.

Trust becomes something you have to prove

Another announcement that caught my attention was the AI Transparency Initiative. That is a much more practical interpretation of AI transparency than simply publishing principles about responsible AI. It is an attempt to create a common approach to making AI models and agents easier for organisations to understand, assess and trust.

We have spent years building architectures primarily for people and applications to consume data. Now we are building for AI agents. Those agents need access to the right data and enough context to understand its relationships and meaning. They also need to know what they are permitted to use. This change makes data architecture and data governance much closer to the AI conversation. A beautifully designed agent sitting on top of poorly understood, poorly governed or inaccessible enterprise data is still going to struggle.


The data layer is changing 

Azure HorizonDB is now generally available, alongside the continued development of Microsoft Fabric. What interested me was the direction rather than an individual product announcement. The data platform is increasingly being designed around AI and agents as consumers of enterprise data, not only people and applications. Microsoft's own summary of the Azure announcements describes the direction as bringing silicon, infrastructure, data and operations together as a system for running production AI and agents at scale. 











Security is becoming agentic

Microsoft Perception was also announced.

The model Microsoft showed is based on three functions: red team to simulate attacks, blue team to detect and triage, and green team to fix and remediate.  Microsoft is applying agentic AI not only to business productivity, but to defending the AI and technology environment itself. 



Ubiquitous intelligence changes governance

Those things still matter, but they are not enough for an environment where intelligence is becoming embedded everywhere. Governance increasingly has to be part of the architecture, not something added around it. An agent needs an identity, access to the right data and clear policies around what it can do. We need to be able to understand the actions it takes and the decisions it makes. There also needs to be transparency around the models behind it, where agents are operating and what information they are using. That visibility is increasingly important if organisations are going to remain accountable for the outcomes their AI produces. The shift towards agentic AI changes the relationship between people, technology and data.

Human ambition, intelligence and trust were the three themes that stayed with me from the keynote.

Human ambition matters because technology should enable what people and organisations want to achieve. The point is not to deploy the largest possible number of agents.

Intelligence matters because an organisation has knowledge, data, relationships, processes and expertise that make its decisions meaningful. AI needs access to that context without the organisation surrendering control of it.

And trust matters because the more autonomy we give AI, the more important visibility, security, transparency and accountability become.

Those three things are connected. Perhaps that was my biggest takeaway from the keynote. We have spent the last few years talking about how organisations adopt AI. What Microsoft Ignite 2026 started to show is what the architecture looks like when AI is no longer an add-on.

References

aka.ms/AIInAction

aka.ms/Ignite26/HorizonDB

aka.ms/MicrsoftIgnite26Blog

aka.ms/AITransparencyInitiative

Book of News 2026

Thursday, 12 November 2026

Microsoft Purview Data Governance is retiring

Microsoft is retiring the data governance capabilities in Microsoft Purview, with support ending on 12 November 2027, and data governance will be fully embedded into the OneLake Catalog in Microsoft Fabric.

Why Microsoft is making the change

AI is forcing a rethink of data governance. We are moving away from governance being something that sits alongside the data estate towards it becoming part of how data is managed and used.

The problems are familiar. Governance becomes fragmented across platforms and teams, controls become difficult to manage consistently, and manual processes struggle as the data estate grows. None of this comes from bad intentions. Data estates simply grow faster than the governance around them.

Putting governance closer to the data makes sense, particularly when that data is increasingly being used by AI as well as people. It makes it easier to use. An AI agent needs to know whether it can use the data, whether it is sensitive, what it means and whether it can be trusted. That makes good governance more important, not less.

Where governance lives now: OneLake Catalog

The OneLake Catalog becomes the home for Microsoft's data governance capabilities, bringing discovery, governance and security closer to the data in Microsoft Fabric. Microsoft is also clear that this isn't limited to data physically stored in OneLake. It can extend to other data through shortcuts, mirroring and supported connectors.

Some of the key capabilities we can already see in OneLake Catalog include:

  • Explore: a searchable catalogue for finding and understanding data, including tables and columns.
  • Govern: insights and recommended actions alongside domains, tags and governance capabilities.
  • Secure: OneLake security roles and permissions, bringing access controls closer to the data.
  • Policies in Fabric: centrally managed policies for controlling how Fabric data and environments are used.
There are also APIs and Copilot capabilities that make it possible to bring governed data discovery into development and AI workflows.


Microsoft Purview tools

Purview is not disappearing. Microsoft is separating data governance from data security and compliance more clearly. Governance moves into Fabric, while Purview continues to provide security and compliance capabilities across the wider estate.

That includes:
  • Information Protection for classifying and labelling sensitive data.
  • Data Loss Prevention for detecting sensitive data and controlling how it is used or shared.
  • Insider Risk Management for identifying and investigating potentially risky activity.
  • Data Security Posture Management for understanding risks around sensitive data, including its use by AI and Copilot.

Fabric OneLake Catalog is not replacing Microsoft Purview Data Governance in a like for like migration. It is reimaging data governance for AI. It is a new architecture for the AI era. 

What to do next

The technology is changing, but the basics of good governance aren't. Someone still needs to know which data matters, who owns it, who should have access and what good quality and trusted data looks like.

I would start with four things:

  1. Understand what you rely on today. Look at how you actually use Purview Data Governance, not just which features are switched on. What processes and integrations depend on it, and what will need to change?
  2. Prioritise what matters. Identify your critical and sensitive data, where it lives and who owns it. There is little value in migrating a catalogue full of data that nobody is actively governing.
  3. Decide what your future model looks like. Work out what OneLake Catalog will govern, where Purview continues to fit and how you will deal with data and governance requirements that sit outside Fabric.
  4. Don't simply recreate what you have today. You have until 12 November 2027. Use that time to simplify the governance model, clarify ownership and think about what needs to change as more of your data starts to be consumed by AI.

This is new technology, so it will take time to settle and mature. But I don't think the important question is simply“How do we migrate from Purview to OneLake Catalog?”

A better question is:“What does our governance model need to look like when both people and AI are consuming our data?”

Reference

A next step in the evolution of the Microsoft Data Governance strategy



Thursday, 29 October 2026

Be Data Confident Webinar

I will be speaking on the Be Data Confident webinar about Can you trust the data behind your AI on 6th November 2026. Discover how better governance, quality, security and compliance can help you build confidence in your AI strategy. A practical webinar exploring what it means to be truly data confident in the age of AI. Together, we'll examine the challenges organisations face as AI adoption accelerates, and the steps required to build confidence in the data powering your strategy.

I will cover:

  • How to assess whether your organisation's data is truly AI-ready.
  • The biggest governance gaps preventing successful AI adoption.
  • How governance, quality, security and compliance work together to create trusted data, using tools such as Microsoft Purview and Onelake Catalog.
  • What data confidence really means in an AI-driven organisation.
  • Real-world examples of how governance enables innovation and confidence across sectors.
  • Your next steps for creating a scalable foundation for trusted AI and becoming data confident.

 

Whether you're under pressure to deliver quick AI wins, exploring new AI use cases or looking to maximise existing investments, this webinar will help you understand the foundations needed for responsible, scalable AI adoption.  

Saturday, 24 October 2026

Data and AI governance in real life

On Thursday 22 October 2026 I spent the day at the Royal Armouries Museum in Leeds. I delivered 3 training workshops in The Bury Lecture Theatre which is a purpose-built, self-contained, air-conditioned lecture theatre over two tiers. It accommodates between 70 to 250 delegates in cinema style seating. 

 

The training workshops introduced participants to the foundations of Data and AI Governance, emphasising how these disciplines shape business behaviour and why modern organisations can no longer afford to treat governance as a peripheral concern. Rather than presenting governance as a theoretical construct, I focused on its practical impact and how it influences decision making, operational resilience, regulatory posture, and the organisation’s ability to deploy AI safely and at scale. The central message I shared was that governance is not a traditional administrative layer but a strategic capability, and embedding it early and consistently is what enables businesses to innovate without losing control.

To bring the discussions to life I used two analogies, one of a stream and how the water travels from a mountain and the regeneration of a central part of Manchester called Holt Town. 

The first analogy was a stream flowing from a mountain, tracing the journey of water as it moves through different environments, is diverted, contaminated, over used, and eventually becomes scarce enough to trigger hosepipe bans. 

The second analogy centred on the regeneration of Holt Town, a historically significant but long neglected part of central Manchester. It was at the centre of the industrial revolution there. By exploring how regeneration requires coordinated planning, clear ownership, sustained investment, and a shared vision across multiple stakeholders, the analogy mirrored the organisational effort required to build and maintain effective governance. 




We ended the day with drinks in the Royal Armouries Museum's Asian & African Gallery showcasing diverse Asian culture.




Friday, 9 October 2026

Apache Ossie (incubating) Could become important for AI and Analytics

Apache Ossie (incubating) is the universal standard for semantic data







Apache Ossie (incubating) is an industry-wide specification effort to standardize how we exchange semantic metadata across analytics, AI and BI platforms, providing a vendor neutral, single source of truth for semantic data. Ossie was previously known as Open Semantic Interchange (OSI). A semantic model is simply a shared definition of business concepts, metrics and relationships that helps people, reports and AI systems interpret data consistently.

As organisations invest in AI, they face a surprisingly familiar problem: different systems often define the same business concepts in different ways.

A metric such as revenue, customer count or profitability can be calculated differently across BI tools, data platforms and AI applications. The result is inconsistent reporting, conflicting answers and reduced trust in both data and AI.

This is the challenge that Apache Ossie (Incubating) is seeking to address. Apache Ossie is an open, vendor-neutral standard designed to allow organisations to define business metrics, dimensions and semantic models once and share them across multiple platforms. Rather than recreating business logic in every tool, the aim is to establish a common semantic language that can be understood consistently across analytics, BI and AI ecosystems. 

Why is this important? Because AI systems do not just need access to data. They need access to trusted business context. An AI assistant may know what data exists, but without clear definitions of what "customer", "revenue" or "active account" actually mean, its answers can still be misleading.

This is where Apache Ossie becomes particularly interesting. By providing a portable semantic model, organisations can help ensure that people, reports and AI agents are working from the same business definitions. The vision is simple: define business meaning once and use it everywhere. 

The initiative is also attracting significant industry support. Microsoft recently announced its commitment to Apache Ossie, describing semantic interoperability as an important foundation for AI-powered analytics and agentic experiences. The goal is to enable semantic definitions to move more easily between platforms while preserving governance and business meaning. Microsoft is working alongside Snowflake and other partners to support the development of the standard. 

For data governance professionals, Apache Ossie is worth watching. Governance has always been about creating trust in data. As organisations increasingly rely on AI, semantic consistency may become just as important as data quality, ownership and stewardship. Apache Ossie represents an attempt to create a common foundation for that trust across the modern data and AI landscape.

Reference 

https://ossie.apache.org/

Tuesday, 6 October 2026

Data Analytics Monthly Forum

Today i spoke at the 75th Data Analytics Monthly Forum with 2 of my collegues Kamila Spencer and Andrew Edge on Building the foundations for trusted AI. We explored the foundations of AI readiness - from trusted data and governance to security, risk and compliance.
 
DATE: Tuesday 6th October 2026 | 12:00pm – 1:00pm
 
We discussed how to create a secure, well-governed foundation for AI with Microsoft Purview. Here was the agenda:
  • Why AI success depends on trusted and high-quality data.
  • The growing importance of AI governance and emerging standards such as the EU AI Act.
  • How data quality impacts AI performance, reliability and outcomes.
  • How Microsoft Purview can help bring governance, security and compliance capabilities together.
  • Practical approaches to identifying and managing AI related risks.
The goal of the session was to help people leave with a clearer view of what your organisation needs to put in place before AI can deliver value safely and at scale.