Secure, Govern and Manage in Microsoft Fabric

In the Microsoft Fabric, the data platform for the era of AI, the complete analytics platform is secured and governed.  Microsoft Purview is a suite of compliance and governance solutions that help enterprises govern their multi-Cloud and multi-platform data estate including governance of Microsoft 365 applications. Microsoft Purview unifies information protection, data governance, risk management, and compliance solutions to enable one place to manage the compliance and governance for the entire data estate. Fabric is partially integrated with third party catalog tools (Power BI) at the moment, Informatica, Callibra and Alation. This is the vision for which Microsoft Purview is a part.

Currently Microsoft Purview Data Catalog support scan of fabrics, Power BI assets, and will be expanded. There is deep integration into Fabric for Purview.  Areas coming soon:

• Purview automatically provisioned and attached to the fabric
• Enables discovery of data assets across the data landscape
• Expansion to the full set of Fabric’s analytics is forthcoming
• Browse and search assets across the hybrid data estate starting with Power BI
• Connect Fabric artifacts with business, operational and location context
• Data Estate Insights added throughout

Purview Data Loss Prevention Policies

This has automatic detection of upload of sensitive data, supporting trigger alerts for admins and customer policy tips for data owners.

Data owners can provide feedback using report a false positive action

Purview Information Protection for Fabric 

This enables the labelling of Fabric items with organisation sensitivity labels used in Microsoft 365 app. 

Sensitivity labels follow the data automatically as it flows from data lake to business users within Fabric to Office

Data Lineage and impact analysis

In Fabric

Endorsements

Data owners can endorse valuable , high quality content in Fabric to create curated sources of truth, increase their discoverability and encourage reuse. Endorsed items get higher visibility in Fabric data discovery experiences. There are 2 options

• Certification - authorised users/groups who can certify data that meets the organisations quality stnadards
• Promotion - content owners can promte data as a means to highlight data they find valuable and recommend for others to use. 

Microsoft Purview hub in Fabric 

This provides insights about the fabric data estate for admins and data owners. It is the entry point to advanced capabilities in Microsoft Purview Portal . 

 

The roadmap includes

• Additional Fabric item types scanned to Microsoft Purview Data Catalog
• Microsoft Purview Data Loss prevention policies for OneLake
• Microsoft Purview policies to enforce access permissions to sensitive data in Fabric

Things covered so far including lineage, endorsement, scanner API integration with Microsoft Purview Information protection, Microsoft Purview Data Catalog, Microsoft Purview data loss prevention policies, and PurviewHub and these are just the beginning.

Further reading and watch the build session

Administration, Security and Governance in Microsoft Fabric

Secure, govern and manage your data at scale in Microsoft Fabric

Microsoft Build 2023 - the era of AI

The keynote shared some amazing technological advances that will change how analytics will work in the future. For me, the biggest change that had to happen, with the ever growing data volumes is the move to one copy of the data. Moving an ever growing volume of data at some point would become impossible, so really excited with todays announcements.

Satya Nadella talked on 5 of the 50+ announcements

• Bringing Bing to ChatGPT
• Copilot extensibility and plugins (to augment AI systems to access APIs, retrieve useful information, perform new computations and safely act on the user’s behalf)
• Azure AI Studio
• Azure AI Safety (Testing, Provenance, Deployment)
• Microsoft Fabric

 Microsoft Fabric, unifying for the era of AI

 

Microsoft Fabric - data analytics for the era of AI

This has four main elements

• Complete Analytics Platform (everything, unified, SaaS, Secured and governed)
• Lake Centric and Open (OneLake, One Copy, Open at every tier)
• Empower Every Business User (Familiar and intuitive, Built into Microsoft 365, Insight to action)
• AI Powered ( Copilot accelerated, GPT on your data, AI-driven insights)

It is one lake , one copy and one security model. This is a massive paradigm shift enabled by AI advancement and will enable competitive advantage. As we all know AI is only as good as your data. This is a SSAS product with frictionless onboarding, instance provisioning with centralized administration. That having tenant-wide governance, centralized security management with built in compliance. There are persona optimized experiences. It is built on open formats, no longer proprietary.

 

There is real separation of compute and storage with one copy for all computers with universal security.  The next level is having shortcuts to the relevant data to enable ease of access. 

The era of the Copilot is here. Reimagining software development with AI. This consideres user experience, application architecture and safety and security.

In summary

Microsoft Fabric Launch Event | May 24-25

Live digital event https://aka.ms/BuildWithAnalytics

Microsoft Build 2023 Book of News provides a roadmap to all the announcements

Introducing Microsoft Fabric: Data analytics for the era of AI

The Fabric Community  

Administration, Security and Governance in Microsoft Fabric

Microsoft Fabric documentation

Introducing Microsoft Fabric

Responsible AI Maturity Models

There has been a new whitepaper from Aether Central UX Research and Education at Microsoft . They published a path to responsible AI by mapping your organisation's goals. The Responsible AI Maturity Model (RAI MM) is a framework to help organizatons identify their current and desired levels of RAI maturity.  This is a map for guidance, not a measurement tool for punitive purposes. 

The RAI MM contains 24 empirically derived dimensions that are key to an organization’s RAI maturity. There are 3 main categories Organizational Foundations,  Team Approach, RAI Practice with  5 levels

The whitepaper The Responsible AI Maturity Model (RAI MM) 

Microsoft published their Responsible AI Standard about a year ago. 

Read more here : 

Microsoft’s framework for building AI systems responsibly 

Microsoft Responsible AI Standard v2 General-Requirements

The umbrella of Microsoft Purview

Microsoft Purview is made up of many tools. It is in fact an umbrella term for many apps to govern, protect and manage the data estate.

Microsoft Purview over laps into many other areas of the Microsoft products and one such area is Microsoft Priva, which is also concerned with an area of data management. Microsoft Purview and Microsoft Priva provide a unified platform to help you comply with data privacy regulations . 

To learn more about this take a look at Microsoft Learn.   Manage data privacy and protection – Microsoft Priva and Purview | Microsoft Learn
 
 

Modern Analytics and Governance at Scale

Microsoft have released a white paper talking about Analytics and Governance (MAG) at Scale to learn how to successfully build an end-to-end analytics ecosystem aligned to datamesh, datafabric, and datahub. Starting with an enterprise data strategy in place, this enables your organization vision and roadmap to accelerate value creation drawing on your data. This lays the groundwork for building the three fundamental pillars of Modern Analytics and Governance (MAG) at Scale: 

1. Enterprise Data Governance: This includes lineage, classification and policy. Then data ordering and moving to rapid access to data
2. Data Management Foundation: Lakehouse as a foundation includes automated data management, ingestion, standardization, quality, metadata registration and access provisioning
3. Domains and Data Products: Autonomous domains provisioned, analytic federations , ingestion and curation for analytics patterns

This is a transformative approach to the modern analytics ecosystem. Capabilities enabled at scale are

Including data governance in solutions has these objectives

• Discover and describe data
• Automatically secure data
• Democratise data access
• Provide data leaders an understanding of their data

To move towards that enterprise data governance state it is important to build a data foundation . The whitepaper mentions four ways to start with this

1. Creating metadata
2. Mapping data assets
3. Cataloging data assets
4. Automating governance and security

The paper goes on to share an implementation plan and much more for Microsoft Purview . 

MVP Summit 2023

It is MVP Summit this week. 

It is week to catch up with friends and learn. I hope everyone at MVP Summit has fun, learns a lot, provides useful feedback and is understanding. Thanks to the team for organising the event. Sad I am not there but I see my name on the board. #MVPBuzz

Data Toboggan - Cool Runnings 2023

Submit a session for Data Toboggan – Cool Runnings, our 3rd summer edition on  Saturday 24 June 2023. 

There are two session lengths for submissions. These are: 

• 45 mins
• 10 mins

The Call for Speakers is open and awaiting your submissions on Azure Synapse related content. If you are unsure what to submit check out our YouTube channel full of past event sessions. We would love to see your sessions, new speakers or experienced speakers, so submit now.

CfS: https://sessionize.com/data-toboggan-cool-runnings-2023/

You Tube: https://www.youtube.com/c/DataToboggan

Web: https://www.datatoboggan.co.uk/