T-SQL Tuesday #144 – Data Governance reimagination

Data governance is a topic that has raised its head again in the last year, with the introduction of a GA service called Azure Purview.  Data governance is not a new topic in the realms of data management.  I think over the last few years data governance has had a focus on meeting data protection law, government legislation and formalised control and standards. Then with data sovereignty issues being everyday considerations within the global market place, storage locations in the cloud and the likes of the General Data Protection Act, data governance has been very focused on meeting legislative requirements.

There has been a substantive cost involved in setting up data governance within an organisation to avoid those heavy fines if a data breach were to occur. I think because of this many organisations compartmentalise personal data systems.

A change in perspective, a reimagination, of data governance is occurring. Data governance is really about ‘data erudition’, showing an interest in learning about the data we have, improving the quality and creating a more productive and trusted data asset. Starting small and incrementing data change in a way that matches the business need, is a way to gain targeted business value. Azure Purview provides this great opportunity for us to start at the beginning and create a data catalog, data inventory, data dictionary and much more in an automated way.  Trusting the data quality, knowing your risks and what data you have sets your business up for success.

My invitation to you for this month’s #tsql2sday is…

I want to invite you to share your experiences on data governance

• The current cost of data governance versus its benefits
• The amazing things data governance has enabled you to achieve or will enable you to achieve in the future
• The potential uses for Azure Purview within your estates and the automated deployment options for that

*** The Rules ***

• Your post must be published on Tuesday 9th November 2021. This counts as long as it’s still Tuesday anywhere in the world.
• Include the T-SQL Tuesday Logo and make it link to this invitation post.
• Please send me a tweet @victoria_holt with a comment to this post with a link to your own so I know where to find it.
• Please tweet about your post using the #tsql2sday hashtag.

Data Erudition

I have been thinking about data governance, Azure Purview and data research. How these areas link together and enable innovation, that democratisation of data and holistically thinking about data as a strategic asset. A combination of above in my diagram covers my thoughts.

Azure World Synapse Day - what inspires and empowers

 

We learn by doing, people sharing experiences and reading. A retrospective for technical and business experience to help bridge the gap to innovative solutions. This conference is to share how Azure Synapse Analytics inspires you, empowers you, and how it accelerates your business analytics. 

Register https://bit.ly/RegisterDTWSD21

PASS Data Community Summit

 

PASS Data Community SUMMIT is November 8-12, 2021 . It is a free online conference and evolved from the original PASS summit. PASS Data Community Summit 2021 is the year’s largest gathering of Microsoft data platform professionals. Join this free online Summit and connect, share, and learn with 1000s of your peers from the global data platform community.

Microsoft Ignite November Edition

Microsoft Ignite is back for the second instalment this year. It is November 2–4, 2021 to explore the latest tools, training sessions, technical expertise, networking opportunities, and more. I like to think of Ignite as empowerment and innovation with end to end experiences. The four transformation curated themes 

• Empower everyone for a new world of hybrid work
• Innovate anywhere from multi-cloud to edge
• Build a hyperconnected business
• Protect everything with end-to-end security

Let innovation drive world Azure Synapse Day 2021

Want some more information and background about our November Conference on 12 November 2021. One of our organisers has written a great post explaining it here.

You can read it here as well

So, while there’s been all this stuff going on, a couple of data friends and I started a conference series and a user group both focused on Azure Synapse Analytics. Because, you know.. you can never have too many online conferences, training sessions, and presentations, right ?

We’ve probably all attended online lerns recently, been to meetings online, chatted with friends and family over new and sometimes unfamiliar apps like Teams and Zoom (yes, the nerds have been using them for a while, but collab apps just went *wild* recently).

But we (as in me and the aforementioned data friends) wanted to do something different. Something more…. interactive. More…. conversational. Less “I’ll sign up and download the recordings later, maybe“. And so, somewhat concietedly, we came up with ‘World Azure Synapse Day 2021’, where we can share, chat, learn, and laugh (maybe) to round out the year. A sort of retrospective on the future, if you like.

Full disclosure: we didn’t come up with the format ourselves. A few months back, a company called DataStax did something similar for the worldwide Apache Cassandra community where they had short talks from the engineering team, the sales people, their marketing. We’re aiming for something waaaayyyy less corporate, and more about you  We just borrowed the good bits and left out the sales pitch.

So we’re doing three sessions on the day, timed to hopefully coincide with people’s availability in each of APAC, EMEA, and AMER time-zones. We hope it will feel a bit more like a meeting than a presentation, as there’ll be several short talks and interviews in each. The sessions will *not* be recorded (sorry, you’re going to have to be there to see it !) – it’s really important to us that each session flows and everyone can participate, and recording that doesn’t seem fair. We’re also not even publishing an agenda, just a schedule of who will be talking at each session – we want the speakers to speak for themselves in every way possible – so there’s no pressure on anyone to ‘get it right first time’, and each and every contribution is welcome.

If you want to attend (and we hope you do), please sign up to the MeetUp for Data Toboggan here. There’s also a code of conduct under the heading ‘Be Excellent To Each Other’ that we’d ask you to adhere to, but that’s all you have to do. If you want to have your voice heard, just submit some thoughts through Sessionize, and we’ll be in contact. And don’t hesitate to ask any question you like in the MeetUp group – we’ll make sure you get a response 

Hope to see you at Data Toboggan – World Azure Synapse Day 2021 ! Until then, stay safe.

The Data Toboggan Team

Azure Purview Generally Available

The maximize the value of your data in the cloud: achieve unified data governance with Azure Purview digital event with Rohan Kumar, Corporate Vice President Azure Data and Mike Flasko, General Manager of Azure Data Governance Platform on 26 September 2021 was an exciting event.  It enables the new reimagined agile data governance world to move forward.

The event explored Azure Purview, a unified data governance solution that gives you a holistic, up-to-date map of your entire data estate. The general availability of Azure Purview was announced bringing that improve automated governance to the fore. 

The product launched with an area in preview. 

There is automated data discovery using Purview data scan supporting hybrid data landscapes with classification and lineage. There are 200+ data classifiers with 35 data sources. The data map graph describes the data assets and relationships across the estate with fine gained access controls. The lineage feature is really important as it enables root cause analysis to happen and ensures data lineage is available via visualisation. Being able to search, browse and curate data enables you gain more understanding of your data. 

Business context with data is important and to have a business glossary enables relevant business terms to be connected. There is hierarchy support and an integrated approval workflow.

There are a large set of connectors that enable data scanning currently available and there is pubic preview of more data options for data scanning for Google Cloud, Erwin, Salesforce, IBM DB2 and Cassandra. Data scanning options coming soon are snowflake, SAP HANA, PostgresSQL, MongoDB and MySQL.

Azure Purview is growing in functionality all the time and Purview Data Insights is in preview. This looks at 

• data asset distribution 
• sensitive data
• data scanning coverage 
• business glossary utilisation

The insights capability is useful for CDOs to gain a high level picture of the data estate.

A glimpse of the exciting roadmap to come was also shared.  

The Azure Purview features are

There a lots of resources already available

Azure Purview overview 

Introduction to Azure Purview Microsoft Learn to start your learning journey.

Official Purview blog for updates

Billing for Azure Purview will start on 1st November 2021. Pricing 

Announcements are detailed here

Customer Stories shared

Microsoft Customer Story-Danish pump manufacturer develops sustainable water solutions with unified data governance from Azure Purview

Microsoft Customer Story-Heathrow boosts operational efficiency and improves decision making with Azure Purview

Microsoft Customer Story-illimity optimizes data governance and streamlines compliance with Azure Purview