Reading UK Regional Spotlight event for Microsoft Build

Today, 25 May 2022, I was at the Microsoft Build regional spotlight event at Reading in the UK.

I had the amazing honour to speak at the event. I was a part of  two very different sessions. 

Where are all the cloud and data engineers? with Richard Conway and Dr Victoria Holt

Photo credit: Craig Porteous

The number of cloud and data engineers currently on the market are too few to satisfy the demand for the global digital transformation agenda. Where are they all? How do we entice new people into this exciting field and better prepare new graduate engineers to use cloud patterns and practices?

In the table topic we talked about the shortage of cloud and data engineers. The possible reasons for this and discussed actionable insights on how business and people can help increase the number engineers and how it is a continuous learning journey.  We answered questions poised by the in person and online attendees. The session was recorded and will be posted for on demand viewing here. 

Gain agility with an integrated data platform with Pragati Jain, Dr Victoria Holt, Richard Conway and Andy Cross

Photo credit: Andy Cutler

How you can accelerate innovation on your application with an integrated data platform that adapts to your needs with Hybrid and multi-cloud capabilities?

We started this ask the expert session discussing the Microsoft Intelligent Data Platform that was announced yesterday and told a story of the apps from the core foundations of data governance, Microsoft Purview, moving through to Azure Synapse to digital twins and Power BI. We answered questions from the in person audience and online attendees on various topics.

The session was recorded and will be posted for on demand viewing here. 

It was a very rewarding day with plenty of opportunity for learning and making new friends. I would like to thank  the other speakers in my sessions for their amazing help,  the amazing tech team, producers, other MVPs , and the audience for selecting our sessions to attend.

Microsoft Build Keynote 2022

 The Microsoft Build keynote announced a raft of new features. Satya took a whistle top of new features within 10 areas.

So many new features but these are the highlights of features i am looking forward to using

Microsoft Intelligent Data Platform is now generally available. This new, integrated platform unifies operational databases, analytics and data governance.  Read about it hear.

The platform accelerates innovation, achieves agility and is built on a trusted platform.

Microsoft Purview, the unified data governance platform Satya said

• Creates a unified map of your data across hybrid sources
• enables discovery of data that powers business insights
• enables insight into sensitive data to be gained across your organization

Power BI Datamarts in public preview

Datamart in Power BI is a new self-service capability included with Power BI Premium that enables users to uncover actionable insights through their own data without any help from IT teams. Power BI integrated governance allows users to apply Microsoft Purview Information Protection sensitivity labels, track data lineage and impact, or retrieve all Datamarts and metadata via automation (scanner APIs).

Public Preview of Azure Synapse Link for SQL . Azure Synapse Link for SQL is an automated system for replicating data from your transactional databases (both SQL Server 2022 and Azure SQL Database) into a dedicated SQL pool in Azure Synapse Analytics. 

Microsoft Purview Data Estate Insights will be generally available July 1, 2022. I will share more details about this in another blog and the new dashboards now available.

The public preview of SQL Server 2022 . This includes link feature for Azure SQL Managed Instance, Azure Synapse Link for SQL, integrated Microsoft Purview and more. Read about it here  

Microsoft Graph is empowering many areas and there are always more Microsoft AI and OpenAI features. There are some interesting metaverse developments. So how to build a metaverse for Data Toboggan is the question?

Microsoft Build book of news  https://news.microsoft.com/build-2022-book-of-news/ 

Microsoft Build Regional Spotlight

It is Microsoft Build this week and this year there are regional spotlights in 5 countries France, Germany, Japan, Latin America and UK.  For the UK tech community the agenda is packed full of different types of  sessions. 

I am involved in a Table Topic session with Richard Conway. This is a live interactive session inspired by community experts to learn best practices and discover helpful resources. 

The session is

Where are all the cloud and data engineers?

Wednesday, May 25 | 1:00 PM - 1:30 PM British Summer Time

This session is hosted in a Microsoft Teams Meeting and will be recorded and posted for on demand viewing on the public event website. The number of cloud and data engineers currently on the market are too few to satisfy the demand for the global digital transformation agenda. Where are they all? How do we entice new people into this exciting field and better prepare new graduate engineers to use cloud patterns and practices?

There are also Ask the Expert sessions to enable attendees to connect with experts to directly have questions answered live. I am helping out as a moderator for a couple of sessions as well.

Making Data Visualisation simple with DevOps practices - Speaker: Ian Pike and  Liping Huang

Wednesday, May 25 | 1:45 PM - 2:15 PM British Summer Time

Gain agility with an integrated data platform - Main Speaker: Pragati Jean, Dr Victoria Holt, Richard Conway and Andy Cross

Wednesday, May 25 | 3:00 PM - 3:30 PM British Summer Time

This session is hosted in a Microsoft Teams Meeting and will be recorded and posted for on demand viewing on the public event website. In this Q&A session, you can learn from our experts on how you can accelerate innovation on your application with an integrated data platform that adapts to your needs with Hybrid and multi-cloud capabilities.

I am looking forward to an exciting day tomorrow with some amazing speakers.

Azure Synapse Influencer program

There is an exciting new program from Azure Synapse called influencers. 

The Azure Synapse Influencer program recognizes Azure Synapse Analytics community members who actively support one another by sharing Synapse-related content, announcements, and product news via social media.

The benefits of the new program

• Official recognition for your support of the Azure Synapse community
• An exclusive badge for each membership tier
• Azure Synapse Influencer spotlights and visibility through official Synapse channels
• Exclusive access to the Synapse engineering team, including events and Q&As
• Content collaboration opportunities with other Azure Synapse Influencers and the Synapse team

It is great to see 3 different tiers of membership.  This is great to add to the content that Data Toboggan helps bring to the community. 

Sign up here.

Follow @Azure_Synapse on Twitter . Share Azure Synapse-related posts and content using the hashtag #SynapseInfluencers and don't forget #datatoboggan for community content

Tech in Ten, On the Road by Venturi!

I was excited to be a part of the Tech in Ten podcast by Venturi discussing SQLBits, Data Governance and Data Toboggan. It has been released as Episode 9.

The Tech in Ten team went down to the SQLBits data platform conference held at ExCeL London. Ben sat down with some of the speakers and data professionals at the event to talk about all things data and the ever-changing industry.

Listen to the podcast on Spotify. 

CDO and Data Leaders Global Summit

Today was the CDO and data leaders global summit hosted by the EDM Council and CDO Magazine designed specifically for senior and C-level data and analytics executives. 
 

One of the sessions was about the EDM. 

There were discussion about the Cloud Data Management Capabilities (CDMC) 14 key controls and Automations. These should help build trust and confidence in the industry. 

The EDM Councils areas of Advocacy 

• Best Practice - DCAM & Cloud CDMC, Data ROI, ESG Data (environmental, social & corporate governance)
• Driving Standards - Knowledge Graph, Industry Ontologies, shared lab
• Training and Certification - virtual & elearning and webinars and events
• Research and Benchmarking - Global Industry Study, Life Sciences, Data Sharing
• Regulatory Engagement - Regulators participation
• Networking - Data Visions, CDO Summit, workgroups and forums

Learn more about the CDMC, and download with a free license for internal use, at: https://edmcouncil.org/page/CDMC

For  those who have been very DAMA Book of Knowledge focused the DCAM (Data Management Capability Assessment Model) is closely aligned. CDMC took the DCAM framework and focused on managing data in Cloud.  It contains a number of best practices specific to the challenges (and opportunities) of Cloud. The council has been in regular communication with global regulators about CDMC, on what controls may be needed in ensuring safe and effective Cloud environments.

Controls that will be used in Snowflake to satisfy the EDMC's CDMC framework can be found at the GIT link is: https://github.com/Snowflake-Labs/EDMC-CDMC-v1-14-Control_Mapping . On the Microsoft side in Microsoft Purview Compliance Manager there are templates for CDMC key controls.

The keynote was interesting talking about how the CDMC - Cloud Data Management Capabilities Framework - will have a global impact as the best practice standard for cloud Data.  The session labelled as global industry standard for accelerating trusted cloud adoption created through global industry workgroup between May 2020 to September 2021.  There were 100+ companies and 300+ participants. The panel was made up of 

 

Two reports to read

EDM Council ESG Corporate Reporting Entities report 

https://app.smartsheet.com/b/form/45e8fb7e57844ff5bb39098733818e70

EDM Council ESG Rating Providers and Data Aggregators report

https://app.smartsheet.com/b/form/37567c73363a4f839f569bbc88406804

Different data models and frameworks

 There are 3 different models that can help when thinking about data governance and data management.

The revised DAMA Wheel has data governance at the top

The core components to think about for data governance and data management are: Policy; Stewardship & Ownership; Culture Change; Strategy; Principles and Ethics; Data Valuation; Data Maturity Assessment; Data Classification.

When getting started I look at the fundamentals as a starting place.

1. Data Governance

2. Meta Data Management

3. Data Quality

4. Reference/Master Data

DAMA-DMBOK | Data Management Body of Knowledge

DCAM (Data Management Capability Assessment Model) was first published in 2014 following the Socratic method of question and debate. . CDMC (Cloud Data Management Capabilities Framework) is a playbook of best practice for managing data in the cloud. Version 1.1.1 was released in September 2021 , created by a cross industry workgroup of 100+ firms. It is a framework for best data management practices to accelerate trusted cloud adoption.  It can be downloaded here. The holistic list of capabilities highlighted in the CDMC from the EDM Council is:

Data Cataloguing and Discovery 

Data Classification 

Data Ownership 

Data Security 

Data Sovereignty and Cross-Border Data Sharing 

Data Quality

Data Lifecycle Management 

Data Entitlements and Access Tracking  

Data Lineage  

Data Privacy 

Trusted Source Management and Data Contracts 

Ethical Use and Purpose 

Master Data Management

Data Management Capabilities Model Framework | Playbook of best practice

The Data Management Maturity (DMM)  program from the CMMI Institute has best practices for providing support for the implementation of process for these five categories: strategy; governance; data quality; operations; and platform and architecture. 

Best practice for ensuring broad participation in the practice and senior oversight of the effectiveness of data management. 

Capability Maturity Model Integration  (CMMI) Six themes  The CMMI models are described as collections of effective practices and process improvement goals that organisations can use to evaluate and improve their processes.