Microsoft Purview Ignite 2022 Announcements

There were several announcements at Microsoft Ignite about Microsoft Purview. 

• ML Based Classification
• Manual Lineage
• Dynamic Lineage for Azure SQL Database
• Metamodels
• Self-service access for Azure SQL Database

The details are

Business Context for your technical data

To enable data to be managed well it is important to have a business lens applied to the data.  To that end Microsoft have added a new capability to enrich the Data Map with business and governance context to make it relevant for the Data Stewards, Chief Data Officers, Data Analysts, and Data Scientists. This new feature is called Purview Metamodel. Three elements are organisation, business processes and data products. This shows how the data is used in business activities, enables the organisational hierarchy to be defined and , defines data used across the business departments and business processes. This is all placed this in the hands of the Data Steward.

To add further enrichment and augmentation to Purview Data Map it provides end-to-end data lineage which the Data Stewards can now annotate so their data in manual data lineage data movement is shown.

Further Reading

Add business context to your hybrid data estate with Microsoft Purview

Microsoft Ignite 2022

The Microsoft Ignite 2022 keynote on 12 October shared many innovations and there were lots of announcements. The Microsoft Book of News October 12 - 14, 2022  details these  https://news.microsoft.com/ignite-2022-book-of-news/

Satya talked about the digital imperative and the world's computer - Azure. The main 5 themes are to do more with less.

The technology world is changing rapidly and Satya mentioned that Gartner predicts that by 2025, 70 % of applications will be made by no code/low code tools, up from 25% in 2020. enabling hand drawn forms for example to be converted into apps with  AI

Microsoft and Databricks deepen partnership for modern, cloud-native analytics

https://techcommunity.microsoft.com/t5/azure-data-blog/microsoft-and-databricks-deepen-partnership-for-modern-cloud/ba-p/3640280  

Microsoft and Databricks have partnered to build a foundation in the Microsoft Intelligent Data Platform by integrating their hallmark capabilities to create an integrated solution for our customers.

Distributed PostgreSQL comes to Azure Cosmos DB

https://devblogs.microsoft.com/cosmosdb/distributed-postgresql-comes-to-azure-cosmos-db/

Azure Cosmos DB for PostgreSQL, a new Generally Available service to build cloud-native relational applications. Azure now offers its own single database service that supports both relational and NoSQL workloads. You can build cloud-native applications for relational and non-relational data using Cosmos DB.

Introducing the Microsoft Intelligent Data Platform Partner Ecosystem

https://techcommunity.microsoft.com/t5/azure-data-blog/introducing-the-microsoft-intelligent-data-platform-partner/ba-p/3640279

There was a launch of a powerful new Partner Ecosystem for the Microsoft Intelligent Data Platform deliver category-leading and cloud-native data and AI solutions integrated with the Microsoft Intelligent Data Platform to complement capabilities to address diverse scenarios.

Data Governance

Data Governance was mentioned at every single opportunity in many sessions. Data Governance looking at security and compliance, data management and responsible democratisation.  Announcements were Microsoft Purview Business workflows (GA), Business metamodel (Preview), Improved root cause analysis and traceability with SQL Dynamic lineage 

More details in the book of news shares:

• Improved root cause analysis and traceability with SQL Dynamic lineage (now generally available) and fine-grained lineage (in preview) on Power BI datasets. You can do thorough root cause analysis from a single location in Microsoft Purview.
• Metamodels that will enable customers to define organization, departments, data domains and business processes on their technical data. This feature is in preview.
• then the Machine learning-based classifications will make detection of human names and addresses simple and scalable in user data. This feature is in preview.

These features with help with big data management, adding context on top of  data, improved classification with AI, manual process lineage and scorecard insights.  You can read about it below

Add business context to your hybrid data estate with Microsoft Purview

https://techcommunity.microsoft.com/t5/security-compliance-and-identity/add-business-context-to-your-hybrid-data-estate-with-microsoft/ba-p/3651989

Customize retention and deletion to help meet your specific business requirements

https://techcommunity.microsoft.com/t5/security-compliance-and-identity/customize-retention-and-deletion-to-help-meet-your-specific/ba-p/3613111

Read the article to learn more about the announcements in the Data Lifecycle and Records Management space that help organizations manage the lifecycle of data.

What's New in Microsoft Purview Compliance Manager

https://techcommunity.microsoft.com/t5/security-compliance-and-identity/what-s-new-in-microsoft-purview-compliance-manager/ba-p/3643375

Compliance manager helps

• Eliminating blind spots with the right set of security, compliance, and privacy controls
• Safeguarding critical data from external and internal threats
• Identifying risks and addressing regulatory compliance requirements

There has been additional automated controls for Microsoft Priva and App Governance announced.

Power BI

Power BI updates Do more with enterprise self-service business intelligence  https://powerbi.microsoft.com/en-us/blog/microsoft-ignite-2022-do-more-with-enterprise-self-service-business-intelligence/ 

with video summary https://www.youtube.com/watch?v=PxGVcFb-zv0&t=28s

PASS Summit 2022 session schedule

 I am pleased to share my session schedule 

Business Benefits of Good Governance

Summit - Day 1 09:30 AM - 10:00 AM PST

Category: Live  Q&A

Theme: Ahead of the curve

Transformation and Innovation: Why the Database Must be Included

with Steve Jones and Joshua Higginbotham

Summit - Day 1 02:30 PM-03:45 PM PST

Category: Panel

Theme: Ahead of the curve

Data Governance with Azure Purview - Ask the Experts ( Now Microsoft Purview)

With Erwin de Kreuk and Wolfgang Strasser

Summit - Day 3 09:30 AM - 10:45 AM PST

Category: Panel

Theme: Revolutionary

Access control in the Microsoft Purview governance portal

Microsoft Purview has a set of predefined roles with differing levels of access defined here. There is always a lot of reading on these pages but further down I spotted a useful chart. Diagrams help show the access clearly.

Metadata Management

Metadata is an important facet of data entities and sits within the data governance space.  Meta data is data about data. There are various categories of metadata.  

• Business metadata describes all aspects used for governance, finding & understanding data.
• Technical metadata describes the structural aspects of data at design time. 
• Operational metadata describes processing aspects of data at run time.
• Social metadata describes the user perspective of the data by its consumers.

Often metadata connects business domains, processes, technology and data. A good place to start is at the technical layer and a tool such as Microsoft Purview can be useful to make this scalable. The end goal is understandability of the data leading to good data quality.

Purview Git Hub Resources

There are a set of solution accelerator on GitHub for Microsoft Purview. These are:

microsoft/Purview-ADB-Lineage-Solution-Accelerator

A connector to ingest Azure Databricks lineage into Microsoft Purview

microsoft/Purview-Machine-Learning-Lineage-Solution-Accelerator

Solution accelerator to help build Machine Learning Lineage

 

microsoft/Purview-Custom-Connector-Solution-Accelerator

Solution Accelerator to help build Purview custom connectors

microsoft/Purview-Custom-Types-Tool-Solution-Accelerator

Solution accelerator for creating custom type definitions in Microsoft Purview.