Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Monday 17 October 2022

Microsoft Purview Ignite 2022 Announcements

There were several announcements at Microsoft Ignite about Microsoft Purview. 

  • ML Based Classification
  • Manual Lineage
  • Dynamic Lineage for Azure SQL Database
  • Metamodels
  • Self-service access for Azure SQL Database

The details are







Saturday 15 October 2022

Business Context for your technical data

To enable data to be managed well it is important to have a business lens applied to the data.  To that end Microsoft have added a new capability to enrich the Data Map with business and governance context to make it relevant for the Data Stewards, Chief Data Officers, Data Analysts, and Data Scientists. This new feature is called Purview Metamodel. Three elements are organisation, business processes and data products. This shows how the data is used in business activities, enables the organisational hierarchy to be defined and , defines data used across the business departments and business processes. This is all placed this in the hands of the Data Steward.




To add further enrichment and augmentation to Purview Data Map it provides end-to-end data lineage which the Data Stewards can now annotate so their data in manual data lineage data movement is shown.

Further Reading

Add business context to your hybrid data estate with Microsoft Purview

Thursday 13 October 2022

Microsoft Ignite 2022


The Microsoft Ignite 2022 keynote on 12 October shared many innovations and there were lots of announcements. The Microsoft Book of News October 12 - 14, 2022  details these  https://news.microsoft.com/ignite-2022-book-of-news/

Satya talked about the digital imperative and the world's computer - Azure. The main 5 themes are to do more with less.









The technology world is changing rapidly and Satya mentioned that Gartner predicts that by 2025, 70 % of applications will be made by no code/low code tools, up from 25% in 2020. enabling hand drawn forms for example to be converted into apps with  AI

Microsoft and Databricks deepen partnership for modern, cloud-native analytics

https://techcommunity.microsoft.com/t5/azure-data-blog/microsoft-and-databricks-deepen-partnership-for-modern-cloud/ba-p/3640280  

Microsoft and Databricks have partnered to build a foundation in the Microsoft Intelligent Data Platform by integrating their hallmark capabilities to create an integrated solution for our customers.



Distributed PostgreSQL comes to Azure Cosmos DB

https://devblogs.microsoft.com/cosmosdb/distributed-postgresql-comes-to-azure-cosmos-db/

Azure Cosmos DB for PostgreSQL, a new Generally Available service to build cloud-native relational applications. Azure now offers its own single database service that supports both relational and NoSQL workloads. You can build cloud-native applications for relational and non-relational data using Cosmos DB.











Introducing the Microsoft Intelligent Data Platform Partner Ecosystem

https://techcommunity.microsoft.com/t5/azure-data-blog/introducing-the-microsoft-intelligent-data-platform-partner/ba-p/3640279

There was a launch of a powerful new Partner Ecosystem for the Microsoft Intelligent Data Platform deliver category-leading and cloud-native data and AI solutions integrated with the Microsoft Intelligent Data Platform to complement capabilities to address diverse scenarios.













Data Governance

Data Governance was mentioned at every single opportunity in many sessions. Data Governance looking at security and compliance, data management and responsible democratisation.  Announcements were Microsoft Purview Business workflows (GA), Business metamodel (Preview), Improved root cause analysis and traceability with SQL Dynamic lineage 













More details in the book of news shares:

  • Improved root cause analysis and traceability with SQL Dynamic lineage (now generally available) and fine-grained lineage (in preview) on Power BI datasets. You can do thorough root cause analysis from a single location in Microsoft Purview.
  • Metamodels that will enable customers to define organization, departments, data domains and business processes on their technical data. This feature is in preview.
  • then the Machine learning-based classifications will make detection of human names and addresses simple and scalable in user data. This feature is in preview.

These features with help with big data management, adding context on top of  data, improved classification with AI, manual process lineage and scorecard insights.  You can read about it below

Add business context to your hybrid data estate with Microsoft Purview

https://techcommunity.microsoft.com/t5/security-compliance-and-identity/add-business-context-to-your-hybrid-data-estate-with-microsoft/ba-p/3651989

Customize retention and deletion to help meet your specific business requirements

https://techcommunity.microsoft.com/t5/security-compliance-and-identity/customize-retention-and-deletion-to-help-meet-your-specific/ba-p/3613111

Read the article to learn more about the announcements in the Data Lifecycle and Records Management space that help organizations manage the lifecycle of data.

What's New in Microsoft Purview Compliance Manager

https://techcommunity.microsoft.com/t5/security-compliance-and-identity/what-s-new-in-microsoft-purview-compliance-manager/ba-p/3643375

Compliance manager helps

  • Eliminating blind spots with the right set of security, compliance, and privacy controls
  • Safeguarding critical data from external and internal threats
  • Identifying risks and addressing regulatory compliance requirements

There has been additional automated controls for Microsoft Priva and App Governance announced.

Power BI

Power BI updates Do more with enterprise self-service business intelligence  https://powerbi.microsoft.com/en-us/blog/microsoft-ignite-2022-do-more-with-enterprise-self-service-business-intelligence/ 

with video summary https://www.youtube.com/watch?v=PxGVcFb-zv0&t=28s