Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Showing posts with label Microsoft Purview. Show all posts
Showing posts with label Microsoft Purview. Show all posts

Thursday, 14 May 2026

From governance frameworks to enforceable control capabilities

For many organizations, the challenge is not a lack of data governance frameworks, but a gap between principles and practice. Discussions around Microsoft Purview often focus on individual features, while governance frameworks such as DAMA, ISO, or emerging AI regulations describe what should exist at a conceptual level. What organizations actually need is a capability‑led view: a clear map that shows which governance needs exist, how those needs are implemented through concrete Purview capabilities, and where accountability typically sits across the business. This capability perspective bridges strategy, regulation, and day‑to‑day delivery turning governance intent into enforceable, operational controls.

The difference in views:

  • Most Purview discussions list features.
  • Most governance frameworks describe principles.

What organizations actually need is a capability map showing:

  • Which governance need exists
  • Which Purview capability supports it
  • Who typically owns it

This table‑driven view bridges strategy, regulation, and day‑to‑day operations.

Microsoft Purview Capability Mapping Table

Governance CapabilityPurview ToolingPrimary Framework AlignmentTypical Accountable Role
Enterprise data discoveryData MapDAMA – Metadata MgmtData Governance Office
Business data understandingUnified CatalogDAMA – Data GovernanceData Owners / Stewards
Metadata managementUnified CatalogDAMA / ISO 38505Data Governance
Data lineageLineageDAMA / AI Act Art.10Data Engineering
Data quality signalsData Estate InsightsDAMA / ISO 8000Data Quality Lead
Sensitive data classificationInformation ProtectionISO / AI ActSecurity & Privacy
Persistent protectionSensitivity LabelsISO / GDPR / AI ActSecurity
Data loss preventionDLPISO / RegulatorySecurity Operations
Insider risk monitoringInsider Risk MgmtISO accountabilitySecurity & HR
AI data risk visibilityDSPMAI ActSecurity & Governance
Audit loggingAuditISO / AI ActLegal & Compliance
Regulatory control mappingCompliance ManagerISO / AI ActRisk & Compliance
Legal investigationseDiscoveryISO / RegulatoryLegal
Retention & disposalRecords MgmtISO / GDPRInformation Management

Why this matters for AI governance

The AI Act does not introduce new governance concepts, it enforces existing ones at AI scale.

Purview’s strength is that:

  • The same sensitivity labels used in email
  • Also govern datasets
  • Also constrain AI interactions
  • Also support legal discovery

This continuity is exactly what auditors and regulators expect.

Common implementation mistake to avoid

Treating Purview as a security tool
Treating governance as policy documentation
Treating AI governance as separate

Treat governance as a cross‑functional operating model and use Purview as the control fabric beneath it.  Thinking of 

  • Frameworks that define intent.
  • Regulation that demands proof.
  • Tools that deliver evidence.

Microsoft Purview sits at the intersection not as a framework replacement, but as the mechanism that allows modern data governance to function at scale.

Saturday, 9 May 2026

How Data Governance Frameworks Converge

From DAMA to ISO to the EU AI Act how Data Governance frameworks converge and how Microsoft Purview operationalises them is important to understand. Organizations rarely struggle because they lack frameworks. They struggle because frameworks remain theoretical while data, AI and regulation operate at scale.

DAMA‑DMBOK, ISO data governance standards, and the EU AI Act all address the same core problem from different angles:

  • DAMA defines what good data management looks like
  • ISO defines how governance should be assured and audited
  • The AI Act defines where governance becomes legally mandatory

Understanding where these overlap and how tooling like Microsoft Purview can operationalise them is now essential for any organization deploying analytics, automation, or AI in production.

DAMA‑DMBOK: The authoritative body of knowledge

DAMA‑DMBOK is a vendor‑neutral reference framework that defines data management as an enterprise capability, with Data Governance at its core. It establishes what must exist, without prescribing technology. [dama.org]

Key DAMA governance expectations

  • Ownership and accountability for data assets
  • Enterprise metadata and lineage
  • Data quality management
  • Security, privacy, and ethical data use
  • Stewardship and domain governance

Critically, DAMA positions metadata, lineage, and quality as foundational the same elements now required by AI regulation and ISO assurance.

ISO standards: Governing data as an accountable asset

ISO standards translate governance principles into assurable controls.

Key standards relevant to data & AI governance

  • ISO/IEC 38505‑1: Governance of data within IT governance
  • ISO 8000: Data quality management
  • ISO/IEC 25642: Data collaboration and controlled data reuse

ISO explicitly frames data as a managed, governed organizational asset that should consider value, risk, and compliance. 

Where DAMA explains what to govern, ISO defines:

  • Who is accountable
  • How governance is monitored
  • How conformance is evidenced

This distinction becomes critical for regulatory audits.

The EU AI Act is when governance becomes mandatory

The EU AI Act, particularly Article 10, legally mandates data governance for high‑risk AI systems. 

Article 10 explicitly requires:

  • Documented data sources and provenance
  • Training, validation, and test data quality controls
  • Bias detection and mitigation
  • Dataset representativeness and contextual relevance
  • Ongoing governance across the AI lifecycle

In effect, the AI Act codifies long‑standing DAMA and ISO principles into law. Non‑compliance now carries legal, financial, and reputational risk.

There is an update to the EU AI Act where EU leaders have agreed to amendments.  The official regulation it is hoped will be passed before the 2 August 2026. A delay of enforcement date has been shared for high-risk AI systems from 2 August 2026 to 2 December 2027 for AI systems listed in Annex III and 2 August 2028 for AI systems covered by Annex I). 

Where the frameworks align

Governance ConcernDAMA‑DMBOKISOEU AI Act
Data ownership & accountability
Metadata & lineage✔ (Article 10)
Data quality management✔ (ISO 8000)
Bias & ethical useEmergingPartial✔ Explicit
Audit & assuranceIndirect✔ Core✔ Mandatory
Lifecycle governance

This convergence means organizations no longer need separate governance programs, they need one operating model that satisfies all three.

Where Microsoft Purview fits

Microsoft Purview does not replace DAMA, ISO, or the AI Act. It operationalises them.

Purview provides:

  • Metadata capture and lineage at scale
  • Policy‑driven classification and protection
  • Evidence‑based compliance reporting
  • Continuous monitoring across data and AI usage

This allows governance teams to move from declared compliance to demonstrable controlDAMA tells you what good looks like. ISO tells auditors how you prove it. The AI Act tells regulators what you must do. The future of data governance is not choosing between these, it is designing one governance model that satisfies all three.

Friday, 8 May 2026

Data governance explained: Tools, pitfalls and how to get it right

Here are the key action points for establishing a successful data governance strategy based on the video "Data governance explained: Tools, pitfalls and how to get it right," 

Align with Business Strategy
​Start with the "Why": Identify the specific business needs, such as compliance (GDPR, AI Act), fundraising, or medical records [04:18].
​Treat Data as an Asset: Data governance must flow from the top of the organization down to the bottom, rather than being treated as a side project [02:21].
​Focus on Use Cases: Define how data governance will help the business specifically—whether it's for better reporting, compliance, or making costly business decisions [03:33].

Establish Roles and Ownership
​Identify Data Owners: Determine who is responsible for your "critical business assets." You cannot govern everything at once, so prioritize the most important data [10:24].
​Appoint Data Stewards: Assign individuals to manage the day-to-day operational quality of the data within their specific departments (e.g., Finance, Product, Service teams) [10:38].
​Create a Governance Model: Set up a central "meeting place" or committee where people from different parts of the business can talk and manage data issues together [09:44].

Build the Foundation (Before Tooling)
​Develop a Business Glossary: Create clear, shared definitions for terms. Different departments often use the same words to mean different things, leading to confusion [05:18].
​Assess Data Quality: Be honest about the current state of your data. Talk to team members to identify which data sets are trusted and which are "bad" [10:49].
​Avoid "Boiling the Ocean": Don't try to govern all 100+ tools at once. Start small with 3-5 business-critical assets and scale from there [12:32].

Implement the Right Tools
​Automate to Scale: Use tools like Microsoft Purview to scan data sets and automate processes that are too large to handle manually [02:54].
​Bridge the Gap: Ensure the technical team (who deploys the tool) and the business users (who use the data) are not disconnected. The tool is only effective if the business processes are mapped into it [03:01].
​Leverage Frameworks: Use established frameworks (like the CDMC framework) to guide your rollout and ensure you are meeting industry standards [11:49].

Foster a Data Culture
​Prioritize Data Literacy: Invest in training so that employees understand the importance of data and how to manage it as part of their daily operations [08:19].
​Be Proactive: Move away from "reactive" governance (only fixing things when they break or for audits) toward a proactive culture where governance is embedded in every project, especially AI [08:39].
​Watch the full video here: 




Sunday, 3 May 2026

Microsoft Purview a Unified Platform

Modern organizations no longer struggle with a lack of data  they struggle with lack of control, visibility, and trust in that data. Data now spans SaaS platforms, cloud analytics services, collaboration tools, AI systems, and on‑prem environments. At the same time, regulatory pressure, security risk, and AI‑driven data reuse continue to increase.

Microsoft Purview addresses this challenge by providing a single, integrated data governance, security, and compliance control plane across the enterprise. Rather than deploying disconnected tools for cataloguing, classification, protection, policy enforcement, investigation, and audit, Purview enables organizations to manage the entire data lifecycle consistently  from discovery and understanding, through protection and monitoring, to legal and regulatory response.

From an executive perspective, the value of Purview is not its individual features, but its ability to:

  • Reduce risk through centralised visibility
  • Enable scale through automation and policy‑driven controls
  • Support innovation and AI adoption without losing governance
  • Provide defensible evidence for regulators, auditors, and boards

Thus Purview allows organizations to move faster with data, safely and to do so using native tooling already embedded across Microsoft 365, Azure, Fabric, and the broader cloud estate. I wanted to share a current state of the tools as there have been many changes of the last couple of years.

Microsoft Purview – Data Governance Tools

The purpose is to understand, trust, and responsibly reuse data across the enterprise. Microsoft Purview’s data governance capabilities focus on metadata, not the data itself. They provide a federated governance model that enables central standards while allowing data ownership to remain close to the business. These are core tools required for AI success.

Data Map

The Data Map scans and inventories data assets across Azure, Microsoft 365, on‑premises systems, and supported multi‑cloud platforms. It captures technical metadata, classifications, and relationships without copying underlying data. From a technical standpoint, the Data Map:

  • Maintains a continuously updated inventory of data assets
  • Supports automated classification during scan operations
  • Acts as the backbone for lineage, catalog, and insight services

Unified Catalog

The Unified Catalog is the business‑facing layer of Purview data governance. It allows users to search, understand, and request access to data using business language rather than technical system names. Key technical capabilities include:

  • Metadata curation and endorsement workflows
  • Business glossary alignment
  • Ownership and stewardship assignment
  • Data quality and health indicators

The catalog does not grant data access itself it integrates with platform security controls to ensure governance without breaking separation of duties.

Data Lineage

Purview lineage provides end‑to‑end visibility of data flows, showing how data moves from source systems through transformations to consumption layers such as analytics or AI models. Technically, this supports:

  • Impact analysis for change management
  • Root‑cause analysis for data quality issues
  • Explainability for analytics and AI outcomes

Microsoft Purview – Data Security Tools

There purpose is to help protect sensitive data dynamically, wherever it lives or moves. Microsoft Purview data security solutions are designed around the principle that data protection must follow the data, not rely solely on perimeter security.

Information Protection

Information Protection enables classification and protection through sensitivity labels that persist with the data. From a technical perspective:

  • Labels can trigger encryption, access restrictions, and visual markings
  • Labels are consistently enforced across Microsoft 365 services
  • Labels integrate downstream with DLP, Insider Risk, and eDiscovery

Sensitivity labels act as the policy anchor for most Purview controls.

Data Loss Prevention (DLP)

Purview DLP enforces policy‑based controls to prevent accidental or intentional leakage of sensitive data across:

  • Email and collaboration tools
  • Endpoints and browsers
  • Cloud applications and AI experiences

DLP evaluates content, user context, and activity in real time to determine policy actions.

Insider Risk Management

This capability correlates user behaviour, activity signals, and data sensitivity to identify potential internal risks. Technically, it:

  • Analyses sequences of risky actions rather than single events
  • Integrates with Information Protection and DLP signals
  • Supports adaptive policy enforcement

Data Security Posture Management (DSPM)

DSPM provides aggregated, AI‑driven visibility into data risk across the estate, including traditional workloads and AI applications. It enables:

  • Discovery of unknown or unmanaged sensitive data
  • Policy coverage gap analysis
  • Prioritised remediation recommendations

Microsoft Purview – Data Compliance Tools

The purpose is to meet legal, regulatory, and internal policy obligations with defensible controls. Purview’s compliance capabilities focus on evidence, monitoring, and response, rather than prevention alone.

Compliance Manager

Compliance Manager maps regulatory requirements (e.g. GDPR, ISO, industry standards) to technical and organizational controls. From a technical view:

  • Controls link to implemented Purview configurations
  • Evidence can be centrally tracked and reported
  • Progress scoring supports audit readiness

Audit

The unified audit log captures user and admin activities across Microsoft services, providing the foundation for investigations and compliance reporting. It supports:

  • Forensic investigation
  • Long‑term retention of activity records
  • Correlation with security and compliance incidents

eDiscovery (Standard & Premium)

eDiscovery enables legal teams to identify, preserve, collect, and review data associated with legal or internal investigations. Technically, it integrates:

  • Sensitivity labels and retention policies
  • Advanced search and review workflows
  • Role‑based access for legal operations

Records & Data Lifecycle Management

These tools manage data retention, deletion, and record declaration based on business, legal, and regulatory requirements. They ensure:

  • Defensible retention policies
  • Automated disposition
  • Reduced data sprawl and risk surface

Microsoft Purview is a data control framework that underpins modern analytics, AI, and digital transformation initiatives. When implemented correctly, Purview allows organizations to:

  • Govern data without slowing delivery
  • Secure data without blocking productivity
  • Prove compliance without manual evidence gathering

That combination visibility, control, and defensibility at scale is why organizations choose an integrated platform rather than isolated tools. Microsoft documentation and architecture descriptions can be found at learn.microsoft.com









Thursday, 30 April 2026

Data Governance explained

I had a very fun packed day in Manchester a few weeks ago talking on my favourite topic Data Governance, AI Governance and Microsoft Purview. Watch my recording here to help you get started.



Wednesday, 8 April 2026

Operationalising Responsible AI: What Microsoft Purview Actually Enables and How to Use It Well

The conversation around Responsible AI is accelerating, but many organisations still struggle with the same practical gap: How do we turn principles into operational behaviour inside real systems?  
Frameworks like GRAICE™ and Microsoft’s Responsible AI Standard set the expectations,  but they don’t tell you how to wire those expectations into your data estate.

This is where Microsoft Purview plays a meaningful, but often misunderstood, role. Purview is not an end‑to‑end Responsible AI lifecycle platform. It doesn’t manage model development, evaluation, or fairness testing. What it does provide is the governance and security foundation that ensures AI systems interact with enterprise data safely, consistently, and in line with organisational policy.

Below are three actionable ways organisations can use Purview to strengthen Responsible AI practice without overstating its scope.

1. Use Purview to establish data boundaries for AI systems
AI systems are only as responsible as the data they can see. Purview’s classification, sensitivity labels, and access policies give organisations the ability to:

- identify sensitive or regulated data  
- prevent AI systems (including Copilot and internal agents) from accessing inappropriate content  
- enforce information barriers and least‑privilege access  
- ensure data minimisation by design  

Why this matters:  
GRAICE™ and Microsoft’s RAI Standard both emphasise data minimisation, privacy, and controlled access. Purview doesn’t enforce RAI principles directly — but it does enforce the data boundaries those principles depend on.

Action:  
Map your AI use cases to Purview sensitivity labels and access policies. Treat this as a precondition for deploying any AI capability.

2. Use Purview’s lineage and scanning to understand AI‑related data risk
Purview lineage is often misunderstood as “AI lifecycle traceability”. It isn’t.  
But it is a powerful mechanism for:

- understanding where sensitive data originates  
- seeing how data flows across systems AI may interact with  
- identifying shadow data sources that could introduce risk  
- supporting DSPM (Data Security Posture Management) for AI workloads  

Why this matters:  
Responsible AI requires organisations to understand the provenance, quality, and risk profile of the data AI systems rely on. Purview provides visibility into the data estate, not the model estate — and that visibility is essential for any RAI programme.

Action:  
Enable automated scanning and lineage for all data sources used by AI applications. Use lineage to identify high‑risk flows before enabling AI access.

3. Use Purview’s AI usage governance to monitor and control how AI behaves with your data
The newest Purview capabilities focus on AI usage governance — including Copilot and internal AI agents. This includes:

- monitoring AI interactions with sensitive data  
- detecting risky prompts or behaviours  
- applying data‑loss prevention controls to AI usage  
- generating audit trails for compliance and oversight  

Why this matters:  
Responsible AI is not just about how models are built — it’s about how they are used. Purview provides the observability and guardrails needed to ensure AI systems behave safely in production.

Action:  
Enable Purview’s AI usage governance features for all enterprise AI tools. Treat AI usage logs as part of your RAI assurance evidence.

In summary Purview does not operationalise Responsible AI on its own — and it shouldn’t be positioned as a lifecycle governance platform.  
What it does provide is the data governance, security, and AI‑usage oversight that Responsible AI frameworks rely on.

If you use Purview to:

1. Set data boundaries for AI  
2. Understand data risk and provenance  
3. Monitor and govern AI usage  

you create the conditions in which Responsible AI can actually function.


Why Data Catalogues Fail (And How Purview Is Quietly Fixing the Industry’s Blind Spots)

Most data catalogues fail for a simple reason: they  assume that documentation alone creates understanding. It doesn’t. A catalogue full of stale metadata, incomplete lineage, and inconsistent tagging is worse than useless and it creates a false sense of confidence. Many organisations have learned this the hard way, investing heavily in catalogues that quickly became digital graveyards.

Purview succeeds where others fail because it treats the catalogue as part of a governance ecosystem, not a standalone tool. Lineage, classification, access policies, and data maps are not optional extras. They are the core of the experience. This integrated approach ensures that metadata is accurate, automated, and actionable.

Another blind spot Purview addresses is operational relevance. Traditional catalogues focus on documentation whereas Purview focuses on control. It doesn’t just describe data as it also governs it. This shift from passive to active metadata is what makes Purview viable at enterprise scale.

Purview also excels in hybrid and multi‑cloud environments, where many catalogues struggle. Its connectors, scanning capabilities, and policy enforcement mechanisms are designed for real‑world estates, not idealised architectures.

Purview is integrated with Fabric which positions it as the governance backbone of the Microsoft ecosystem. As organisations consolidate their data platforms, Purview becomes the source of truth that ties everything together.



Saturday, 28 March 2026

Series Index Summary: Data Governance, Purview, and Responsible AI

This four‑month series explores the shifting landscape of data governance, Microsoft Purview, and Responsible AI at a moment when organizations are being forced to rethink how they manage, understand, and trust their data. Across the posts, the series traces a clear arc: from the maturing of governance in 2025, through the practical realities of Purview adoption, to the cultural and architectural shifts required to lead in the age of AI.

The December posts set the stage by examining why governance finally became a strategic priority, how Purview’s quieter updates are reshaping the platform, and why AI risks making organizations intellectually complacent without strong data foundations. These pieces frame governance not as bureaucracy, but as the mechanism that makes innovation safe.

I move deeper into strategy and Responsible AI. It explores the predictions shaping 2026, the operational implications of Microsoft’s updated Responsible AI framework, and the evolution of Purview’s classification engine. The AI Is Making Us Dumber series continues here, highlighting the risks of over‑automation and the importance of maintaining human understanding.

I shift into technical depth and organizational reality. It covers SQL Server’s new direction, the strategic value of metadata, and a detailed breakdown of Purview’s February feature updates. The month closes with reflections on why organizations struggle to operationalize policy and how governance must adapt to keep pace with rapidly learning AI systems.

March brings the series to a forward‑looking conclusion. It introduces the concept of contextual governance, examines the architectural convergence of Fabric and Purview, and challenges data leaders to unlearn outdated assumptions. These posts emphasize that leadership in the AI era requires adaptability, transparency, and a willingness to rethink long‑held beliefs.

Together, these posts form a cohesive narrative about where data governance is heading, what Purview is becoming, and how organizations can navigate the accelerating complexity of AI‑driven data estates. I wanted to add clarity in a landscape full of noise and understand that governance is no longer optional, but foundational.

Wednesday, 25 March 2026

Unifying the Data Estate for the next AI Frontier Fabcon Keynote

The Atlanta FabCon keynote was delivered last Wednesday by Amir Netz (CTO and Technical Fellow), Arun Ulag (President, Azure Data), Shireesh Thota (Corporate Vice President, Azure Databases).  It has was recorded. You can watch it here

Session Abstract

As organizations race to deploy generative and agentic AI, the biggest challenge they face is not models, it’s their data estate. Join Microsoft engineering leadership to learn how Microsoft’s databases can be unified through Microsoft Fabric and OneLake, creating a single, governed foundation for analytics, AI, and intelligent agents. Discover why this shift represents a fundamental change in how modern data platforms are built, managed, and scaled for the next AI frontier.

A summary of the announcements.





Friday, 20 March 2026

Safeguarding the AI Frontier with Microsoft Purview & Fabric Innovations

The speed of AI transformation is accelerating. However, for many organizations, that speed is throttled by a critical concern, that of Data Governance. At the Microsoft Fabric Community Conference this week, Microsoft unveiled a suite of innovations designed to bridge the gap between rapid AI adoption and robust data security. By deepening the integration between Microsoft Purview and Microsoft Fabric, they are providing a secure-by-design foundation for the AI era.
Here is a breakdown of the major announcements.

Data Security: From Protection to Prevention

In an AI-driven world, data oversharing is a primary risk. Microsoft is addressing this by extending Purview’s sophisticated security controls directly into the Fabric ecosystem.

Information Protection Policies: Security admins can now define policies in Purview that automatically enforce access permissions based on sensitivity labels. If a file is labeled Highly Confidential, Fabric respects those boundaries automatically.
 
Data Loss Prevention (DLP) for Fabric is now in preview, DLP policies can identify sensitive information (like SSNs or credit card numbers) as it is uploaded to Fabric. This allows for automatic risk remediation, preventing data leaks before they happen.
 
Trusted Workspace Access allows for secure connections to data sources behind firewalls, ensuring that OneLake remains a secure environment even when pulling from complex network topologies.

Risk Management: Visibility into the Human Element

Governance isn't just about locking down files; it is about understanding user behavior.
 
Purview Insider Risk Management is one of the most significant announcements integrating with Fabric. This allows organizations to detect, investigate, and act on potentially malicious or inadvertent activities, such as mass data downloads or unauthorized sharing directly within the Fabric environment.

Data Discovery & Curation: The Reimagined Governance Experience

Microsoft is moving away from the policing model of governance toward an enabling model. They’ve introduced a reimagined governance experience that is business-friendly and federated.

Unified Catalog & Metadata: Fabric’s built-in metadata and lineage are now seamlessly reflected in Purview. This gives a single pane of glass view across your entire multi-cloud estate, making it easier for users to find the data they need while ensuring it meets quality standards.

Item Tagging is a new feature allowing users to add tags to Fabric items. This significantly enhances discoverability and encourages the reuse of high-quality data assets across the organization.

AI Readiness: Building on a Trusted Foundation

The ultimate goal of these updates is AI Transformation. You cannot have a reliable Copilot if it is grounded in unverified or ungoverned data. By automating discovery, classification, and protection, Microsoft Purview ensures that the data fueling your AI models is:
 Accurate: Through better curation and quality checks.
 Compliant: Adhering to regional and industry regulations.
 Secure: Only accessible to the right people (and the right AI agents).

Final Thoughts

The announcements from FabCon 2026 signal a shift. Data governance is no longer a hurdle to be cleared. It is the engine that allows AI to run safely. For those of us managing data estates, the tighter synergy between Purview and Fabric offers a clear roadmap to innovate with confidence.


Saturday, 14 March 2026

Fabric, Purview, and the New Shape of Enterprise Data Architecture

Fabric has reshaped the Microsoft data landscape by unifying analytics, engineering, and storage into a single experience. But unification alone does not create coherence. The real transformation happens when Fabric is paired with Purview. Together, they form an architecture where data movement, governance, and analytics operate as one system rather than disconnected components.

This convergence matters because modern data estates are too complex to govern manually. Data flows across pipelines, notebooks, semantic models, and AI workloads. Without integrated governance, organisations end up with pockets of visibility rather than a complete picture. Purview provides the lineage, classification, and policy enforcement that Fabric alone cannot deliver.

One of the most powerful aspects of this integration is the alignment between data products and governance. Fabric encourages teams to think in terms of products that are curated, reusable assets with clear ownership. Purview reinforces this by providing the metadata, stewardship, and controls that make data products trustworthy. Governance becomes part of the product lifecycle, not an afterthought.

This new architecture also supports hybrid and multi‑cloud realities. Many organisations are not all using Fabric, nor should they be. Purview’s ability to govern across environments ensures that Fabric becomes a strategic hub rather than a silo. The result is an architecture that is unified, not monolithic but flexible.

As organisations modernise their estates, the combination of Fabric and Purview will become the default pattern. It is not just a technical alignment but it is a governance first architecture for the AI era.

It is FABCON and SQLCON in Atlanta March 16 - 20, 2026. 



Sunday, 8 March 2026

Purview and OneLake Govern tab change

Some of the governance insights previously surfaced in Purview Hub in Fabric  now also appear in the OneLake Catalog’s Govern tab .  The change helps bring governance closer to where the data actually lives, rather than leaving them in a parallel experience that always that wasn't as helpful as it could have been. In the Govern tab, you now see the same posture summaries, recommended actions, and learning resources that were in Purview Hub, but framed within Fabric’s unified governance model. It is a cleaner, more coherent way of surfacing core information about the health of your data estate.

To clarify the purpose of the tools

OneLake Hub = user facing discovery and access

Purview Hub  (in Fabric) = admin facing governance oversight 

Microsoft Purview (external portals) = actual governance,  policy , catalog work

Functionally, the Govern tab now gives you a consolidated view of governance status, recommended actions, sensitivity and endorsement insights, and links into deeper governance tooling. You can drill into items that need attention, track improvements over time, and understand how your organisation is using Fabric’s governance features. The experience also ties directly into the OneLake catalog, so governance isn’t an afterthought. It is embedded in the same place you explore, classify, and manage data assets.

Microsoft hasn’t yet published a formal retirement date for the Purview Hub. Fabric is now  presenting a single, coherent story about how organisations should understand and manage their data estate.






You can learn more about it here.

Saturday, 20 December 2025

From Fabric to Purview: Why Unified Data Estates Still Need Unified Accountability

Microsoft Fabric has given organisations a unified data platform, but unification without accountability is just consolidation. Bringing data into one place doesn’t automatically make it trustworthy, compliant, or well‑governed. The real challenge is ensuring that governance keeps pace with architectural simplification. This is where Purview becomes indispensable.

Purview provides the policies, lineage, and controls that Fabric alone cannot. Fabric unifies the experience; Purview unifies the accountability. Together, they create a governance‑first architecture where data movement, analytics, and AI operate within a controlled, transparent framework. Without Purview, Fabric risks becoming another centralised platform with decentralised chaos.

The organisations that succeed with Fabric and AI, will be those that treat governance as part of the architecture, not an afterthought. Governance must be embedded into pipelines, workspaces, and data products and not bolted on later. When governance is integrated, Fabric becomes a strategic asset rather than a technical convenience.

The future of unified data estates is not just about consolidation, it’s about coherence. And coherence requires governance. This is why Data Governance has always been critical but its value is only just being realised. 







Monday, 8 December 2025

November changes in Microsoft Purview’s Quiet Revolution

Microsoft’s November Purview updates delivered foundational improvements that strengthen the entire governance ecosystem. These weren’t headline‑grabbing features; they were the kind of enhancements that quietly transform day‑to‑day operations. Better lineage depth, more accurate classification, and expanded policy automation all signal a platform maturing in exactly the right direction.

One of the most significant changes was the refinement of lineage visualisation. The new depth controls and clearer dependency mapping make it far easier to understand how data flows across complex estates. This matters because lineage is no longer a “nice to have”—it’s the backbone of responsible AI, regulatory compliance, and operational trust. When lineage improves, everything improves.

Classification also received meaningful upgrades. The engine is becoming more context‑aware, reducing false positives and improving sensitivity detection. This is essential for organisations trying to scale governance without drowning in manual tagging. Better classification means better policies, better access control, and better risk management.

Policy automation saw subtle but powerful enhancements as well. The ability to apply more granular rules across hybrid environments is a game‑changer for organisations with sprawling estates. Governance becomes less about manual enforcement and more about intelligent, automated control.













Tuesday, 15 July 2025

Microsoft Data Platform MVP 2025

I am incredibly thankful for the Microsoft Most Valuable Professional Data Platform Award this year. It has been an incredibly tough year caring for my very ill mother, who has sadly now died. Being able to try and help the community has been the one thing that has given me purpose whilst caring for mum. I know she would be so proud and pleased that I have been been recognised with this award. My grateful thanks go to Microsoft for the award and the community with whom I work.  



Monday, 7 July 2025

Data Toboggan Cool Runnings 2025

Data Toboggan Cool Runnings is on Saturday 12 July 2025. Our first Cool Runnings summer event was in 2020. This was one track. Today we have 3 tracks of talks over a 12 hour period.

 

Our Piste Maps are out with our agenda

Bramberg



Pradaschier



Rigi Kaltbad



Agenda: https://bit.ly/DTCR2025-Agenda

Register now: https://bit.ly/DTCR2025-Register 

Monday, 7 April 2025

Why Fabric’s New Health & Quality Capabilities Matter More Than Ever

At FabCon this year, Microsoft doubled down on something many of us in data governance have been saying for a long time: trustworthy data doesn’t happen by accident. It is engineered, monitored, and continuously improved. The newly announced health, quality, and observability capabilities in Microsoft Fabric signal a decisive shift away from reactive firefighting and toward proactive, platform‑level assurance.

For organisations scaling AI, analytics, and operational data products, this matters. Data Quality and Observability are no longer “nice to have”; they are the minimum viable conditions for responsible, repeatable, and compliant data use.

Below is a concise, actionable breakdown of what these new capabilities mean—and how to turn them into immediate value across your estate.

1. Treat Data Health as a First‑Class Operational Signal

Fabric’s expanded health management capabilities give teams something they’ve historically lacked: a unified, platform‑native view of data system health. Instead of stitching together logs, alerts, and manual checks, you now get:

- Integrated telemetry across pipelines, workloads, and storage  

- Early‑warning indicators for degradation, drift, or failure  

- Operational insights that connect system behaviour to business impact  

This elevates data health from a technical afterthought to a governance‑aligned operational metric. For leaders, it means you can finally answer the question: “Is our data estate healthy enough to trust today’s decisions?”

Action: Establish a weekly “Data Health Review” ritual—short, structured, and tied to business outcomes. Treat it like you would a security posture review.

2. Use Data Quality as a Contract, Not a Cleanup Exercise

The new Fabric capabilities reinforce a principle I advocate in every governance programme: quality must be defined, measured, and enforced at the point of creation.

With Fabric’s enhanced quality tooling, teams can now:

- Define expectations (validity, completeness, timeliness) as part of the data product  

- Monitor quality continuously, not periodically  

- Surface issues directly to producers and consumers  

- Build trust signals into downstream AI and analytics workloads  

This shifts quality from reactive cleansing to proactive assurance , a contract between producers and consumers.

Action: Publish a lightweight “Quality Contract” template for all critical data products. Keep it simple: purpose, expectations, checks, and escalation paths.

3. Make Observability the Backbone of AI Governance

As AI workloads scale, observability becomes the difference between responsible innovation and uncontrolled risk. Fabric’s new observability features support:

- Traceability from source to model  

- Lineage‑aware debugging  

- Impact analysis when upstream changes occur  

- Evidence trails for audits, compliance, and Responsible AI reviews  

This is not just operational hygiene, it is AI governance in practice. You cannot assure fairness, accuracy, or safety in AI systems without deep visibility into the data that feeds them.

Action: Integrate Fabric observability outputs into your Responsible AI lifecycle checkpoints—especially model validation and change‑control reviews.

In summary the message from FabCon is clear: health, quality, and observability are now strategic capabilities, not technical chores. For organisations building modern data estates and especially for those embracing AI, where these features are the foundation of trust.









Tuesday, 1 April 2025

Microsoft Community Fabric Conference 2025 announcements

There were lots of announcements during the FabCon 2025 conference. Here are a few to catch up on. The main Microsoft blog post can be read here: Fueling tomorrow’s AI with new agentic capabilities and security innovations in Fabric

Fabric Data Factory

Fabric Data Factory enables data ingestion through applying data transformations, and orchestrating different data-related activities for modern data management architectures. There were many new features added such as 

  • General availability for CI/CD capabilities in Data Factory 
  • SPN auth available for pipeline CRUD APIs to secure API apps without needing to use their user token when using the REST API.
  • Lakehouse connector to read delta tables, being able to use the information from the deletion vectors to exclude deleted records.
  • Support for column mapping, and automated auto-creation of tables with new schemas.



Further reading Fabric Data Factory: What’s New and Latest Roadmap

There were also many product improvements and features for Mirroring in Fabric

Mirroring in Fabric – What’s new

Migration Assistant for Fabric Data Warehouse

A really exciting announcement to help migrate from Azure Synapse Analytics was unveiled. This Migration Assistant for Fabric Data Warehouse will soon be in preview! The Migration experience is built natively into Fabric and enables the seamless transition from Azure Synapse Analytics (Data Warehouse) to Microsoft Fabric.

Migration Assistant for Fabric Data Warehouse (Preview)

Copilot for Data Engineering and Data Science

There is a new version of Copilot for Data Engineering and Data Science, a next-generation AI-powered assistant designed to elevate data analysis and coding experience. 

Further reading Enhancing AI productivity in Fabric notebooks with Copilot updates

Copilot F2 and above SKUs 

This is great news for customers on F2 and above SKUs that they will have access to Copilot, Fabric data agents, and more.

hashtag

Azure AI Foundry

Organizations can use Azure AI Foundry to connect customized, conversational agents, created in hashtagMicrosoft Fabric.

Microsoft Purview

There are several new innovations in hashtagMicrosoft Purview for protected, AI-ready data
1. Enhancing Microsoft Purview Data Loss Prevention (DLP) support for lakehouse in Fabric to help prevent sensitive data loss by restricting access
2. Expanding DLP policy support for additional Fabric items such as KQL databases and Mirrored databases to show users notification through policy tips when they are working with sensitive data
3. Microsoft Purview for Copilot in Fabric
4. Data observability, now in preview, within Microsoft Purview Unified Catalog

These are a few of the announcements that were of particular interest to me but there were a lot of announcements showcasing the sheer volume of change and growth of Microsoft Fabric.