Let innovation drive world Azure Synapse Day 2021

Want some more information and background about our November Conference on 12 November 2021. One of our organisers has written a great post explaining it here.

You can read it here as well

So, while there’s been all this stuff going on, a couple of data friends and I started a conference series and a user group both focused on Azure Synapse Analytics. Because, you know.. you can never have too many online conferences, training sessions, and presentations, right ?

We’ve probably all attended online lerns recently, been to meetings online, chatted with friends and family over new and sometimes unfamiliar apps like Teams and Zoom (yes, the nerds have been using them for a while, but collab apps just went *wild* recently).

But we (as in me and the aforementioned data friends) wanted to do something different. Something more…. interactive. More…. conversational. Less “I’ll sign up and download the recordings later, maybe“. And so, somewhat concietedly, we came up with ‘World Azure Synapse Day 2021’, where we can share, chat, learn, and laugh (maybe) to round out the year. A sort of retrospective on the future, if you like.

Full disclosure: we didn’t come up with the format ourselves. A few months back, a company called DataStax did something similar for the worldwide Apache Cassandra community where they had short talks from the engineering team, the sales people, their marketing. We’re aiming for something waaaayyyy less corporate, and more about you  We just borrowed the good bits and left out the sales pitch.

So we’re doing three sessions on the day, timed to hopefully coincide with people’s availability in each of APAC, EMEA, and AMER time-zones. We hope it will feel a bit more like a meeting than a presentation, as there’ll be several short talks and interviews in each. The sessions will *not* be recorded (sorry, you’re going to have to be there to see it !) – it’s really important to us that each session flows and everyone can participate, and recording that doesn’t seem fair. We’re also not even publishing an agenda, just a schedule of who will be talking at each session – we want the speakers to speak for themselves in every way possible – so there’s no pressure on anyone to ‘get it right first time’, and each and every contribution is welcome.

If you want to attend (and we hope you do), please sign up to the MeetUp for Data Toboggan here. There’s also a code of conduct under the heading ‘Be Excellent To Each Other’ that we’d ask you to adhere to, but that’s all you have to do. If you want to have your voice heard, just submit some thoughts through Sessionize, and we’ll be in contact. And don’t hesitate to ask any question you like in the MeetUp group – we’ll make sure you get a response 

Hope to see you at Data Toboggan – World Azure Synapse Day 2021 ! Until then, stay safe.

The Data Toboggan Team

Azure Purview Generally Available

The maximize the value of your data in the cloud: achieve unified data governance with Azure Purview digital event with Rohan Kumar, Corporate Vice President Azure Data and Mike Flasko, General Manager of Azure Data Governance Platform on 26 September 2021 was an exciting event.  It enables the new reimagined agile data governance world to move forward.

The event explored Azure Purview, a unified data governance solution that gives you a holistic, up-to-date map of your entire data estate. The general availability of Azure Purview was announced bringing that improve automated governance to the fore. 

The product launched with an area in preview. 

There is automated data discovery using Purview data scan supporting hybrid data landscapes with classification and lineage. There are 200+ data classifiers with 35 data sources. The data map graph describes the data assets and relationships across the estate with fine gained access controls. The lineage feature is really important as it enables root cause analysis to happen and ensures data lineage is available via visualisation. Being able to search, browse and curate data enables you gain more understanding of your data. 

Business context with data is important and to have a business glossary enables relevant business terms to be connected. There is hierarchy support and an integrated approval workflow.

There are a large set of connectors that enable data scanning currently available and there is pubic preview of more data options for data scanning for Google Cloud, Erwin, Salesforce, IBM DB2 and Cassandra. Data scanning options coming soon are snowflake, SAP HANA, PostgresSQL, MongoDB and MySQL.

Azure Purview is growing in functionality all the time and Purview Data Insights is in preview. This looks at 

• data asset distribution 
• sensitive data
• data scanning coverage 
• business glossary utilisation

The insights capability is useful for CDOs to gain a high level picture of the data estate.

A glimpse of the exciting roadmap to come was also shared.  

The Azure Purview features are

There a lots of resources already available

Azure Purview overview 

Introduction to Azure Purview Microsoft Learn to start your learning journey.

Official Purview blog for updates

Billing for Azure Purview will start on 1st November 2021. Pricing 

Announcements are detailed here

Customer Stories shared

Microsoft Customer Story-Danish pump manufacturer develops sustainable water solutions with unified data governance from Azure Purview

Microsoft Customer Story-Heathrow boosts operational efficiency and improves decision making with Azure Purview

Microsoft Customer Story-illimity optimizes data governance and streamlines compliance with Azure Purview

National AI Strategy

The National AI Strategy was released in September 2021. It is a 10 year plan to transform and reshape our society.

The 3 aims are  to

• Invest and plan for the long-term needs of the AI ecosystem
•  Support the transition to an AI-enabled economy
•  Ensure the UK gets the national and international governance of AI technologies right 

The  document contains a roadmap and details of the pillars . It has 3 pillars

Pillar 1 Investing in the log term needs of the AI ecosystem

Pillar 2 Ensuring AI benefits all sectors and regions

Pillar 3 Governing AI effectively

Central Digital and Data Office (CDDO)

The CDDO has been created within the Cabinet Office to consolidate the core policy and strategy responsibilities for data foundations. They will work with partners to improve government’s use and reuse of data to support data-driven innovation across the public sector.

The UK's National AI Strategy

Microsoft Research Summit 2021

 

The Microsoft Research Summit is open to everyone! October 19 - 21, with over 150 sessions across 16 tracks, provides the global research community with an opportunity learn from experts pushing the frontiers of technology. Register now: https://aka.ms/AAdv93n The event will start in three broadcast regions (China Standard Time, British Summer Time, and Pacific Time). Microsoft say

For 30 years, our research community at Microsoft has worked across disciplines, institutions, and geographies to envision and realize the promise of new technologies for Microsoft and for society. Today, we’re inviting the global science and technology community to continue this exploration—because ensuring that future advancements benefit everyone is up to all of us.

Join us at the inaugural Microsoft Research Summit, streaming virtually across three time zones. You’ll have the opportunity to hear from science and technology leaders from around the world—people who are driving advances across the sciences and pushing the limits of technology toward achieving a meaningful impact on humanity.

They want to build a place where research thinks of sustainability, ethics, diversity and is inclusive of everyone. There are some really interesting topics under discussion. 

Data Toboggan World Azure Synapse Day

We are back for another edition of Data Toboggan, but we'd like to do something a bit different this time round.

In the last year, Azure Synapse has celebrated its second birthday, and we've all been busy doing awesome stuff with the capabilities of the platform and loving every minute.

No ? Doesn't sound like your experience ? Or maybe we're bang on the money and it's been awesome ? So, tell us about it !

We want *you* to tell us how Azure Synapse Analytics inspires you, empowers you, and how it accelerates your business analytics. Want to tell us of a not-so-epic experience instead ? Yep, come and do that. Tell us your fails and wins, and what you're hoping for in the coming 12 months of Azure Synapse...

So, this next edition of Data Toboggan won't be a full-day, tech-focussed event.

Instead, we're planning on doing three 1 to 2 hour community sessions where we listen to you tell your story in a more open format.

We're not looking for long sessions - ideally each speaker would get 10 to 15 minutes to share and discuss. We're targeting Friday 12th November as the delivery date, with 3 sessions running at locally convenient times for APAC, EMEA and AMER (times are GMT, because that's where we are, and we hope they translate to 'convenient').

APAC - 08:00 GMT

EMEA - 12:00 GMT

AMER - 17:00 GMT

Look out for submission links heading your way soon ! You can submit to talk at any of the sessions - whichever is most convenient for you.

Thanks for being with us this year. We hope you'll enjoy sticking around for a while.

Stay safe,

The Data Toboggan Team. 

 

Big Data LDN 2021 - Data Governance and the essential CDO

Big Data LDN has been running 22 -23 September. There have been some really interesting sessions covering data governance and the CDO role.

Robin Sutara spoke on the analytics paradox - balancing competing demands for agility and governance. Think big, one technology is not a magic bullet and learn to fail fast. 

Start small with one business problem and build upon that. The data quality with adapt and improve to meet needs of each problem being addressed. It is people and culture that change a business. We need to think beyond the technology. 

 

Great session and an important thought for the future to stop talking architecture and focus on business value. It goes beyond economics and needs sustainability.

Data is a hot mess, so lets cook session from Ben Schein mentioned a key point Data Governance is never done and always be open to new ideas. Data governance is a team sport. You need to find a balance between innovation and experimentation, data teams and consistency, usability and scale. He finished with some important areas to consider

• keep data definitions consistent
• don't wait for  perfect data set to arrive
• data quality is essential for good insights
• let people go shopping for the data they needs
• there is no data magic wand
• data governance is a team sport

Another very insightful session from Cindi Howson on data-driven culture: is your organisation a laggard or leader, gave perspectives on the biggest challenges to being data driven. 

Perspectives on the biggest challenge to being data driven is culture (67%) followed by talent and people (22%).

 

To disrupt your culture bring in a change agent, identify relevance (WIIFM) and organise for collaboration.

She talked about CDOs having a short tenure in organizations being an average of 2.5 years. There is an interesting article about that , Why Do Chief Data Officers Have Such Short Tenures? by Tom Davenport, Randy Bean, and Josh King 

Why WIIFM (what's in it for me) for communication, incentives, skills and tribes and role models.

To move forward into this new world there needs to be a transition in organizational design, from that traditional BI centre of  excellence to this embedded design .

Centralize for economies of scale for common data, infrastructure and specialist talent but decentralize when business domain experts are essential to analytics workflow. Ensure that cross boundary communication is ongoing for best practice, synergies and career management. We are looking at a new hybrid model that is transparent and optimized.

She finished with sharing there is a Data Chief Community (TheDataChief.com) . It has podcasts, blog,  roundtables and community newsletter. 

What we learned from 400 data leaders in CDO summer school  session by Carruthers and Jackson highlighted

• Data strategy is multi dimensional not linear
• Governance is being revised but you need to tell the story. It is still a problem but it needs communicating the purpose through strategy.
• They had Scott Taylor talk about data storytelling - keep simple
• Risk - listen, listen and listen more 
• Soft skills are important
• Data Literacy is a spectrum.
• Nurture your community
• Play to your strengths and culture is the biggest hurdle
• Keep policy simple
• There should be a call to arms on Ethics
• Methods can be disrupted, innovative and evoke change
• It is not about technology outcomes

In the summer school they discussed the DIKW Pyramid which represents the relationships between data, information, knowledge and wisdom

Exasol have published the journey to the CDO. 

https://www.thedatadreamer.com/insights/the-journey-to-CDO/

Data Governance Podcast

 

It was amazing to take part in my first podcast about the benefits of Data Governance and how to get started in Coeo Conversations with Justin Langford.  

The data field is such an exciting place to be at the moment. Data Governance is more than just compliance, it is about managing the whole ecosystem. I thought I would run out of things to say talking for 30 minutes in the podcast on governance, however that was not the case. I hope you find the podcast interesting, informative and fun.

Looking forward there is an exciting digital event coming up Maximize the Value of Your Data in the Cloud: Achieve unified data governance with Azure Purview . The event is Tuesday, September 28, 2021 | 9:00 AM-10:00 AM Pacific Time (UTC-7) Register here