Data Relay Broadcast 2021 Next week

 

Exciting news Data Relay is not only back but starts in 7 days. 5 days of sessions, 10 great speakers - join us April 19th-23rd for a virtual tour of five old English kingdoms.

Monday 19/04/2021 – Wessex

John Martin Infrastructure as Code: An introduction to Terraform

Anna Wykes Custom Logging within DataBricks

Register for Wessex at Eventbrite

Tuesday 20/04/2021 – Mercia

Laura Graham-Brown  Brilliant Report, can I print it?

Arun Sirpal  Azure SQL Database

Register for Mercia at Eventbrite

Wednesday 21/04/2021 – Northumbria

Chris Taylor  Kubernetify your Containers

Kevin Feasel  Does this look weird to you? An introduction to Anomaly Detection

Register for Northumbria at Eventbrite

Thursday 22/04/2021 – East Anglia

Tracy Boggiano Mental Health and Wellness in IT: Safeguarding our most precious resource

Mark Broadbent Edge of Tomorrow, An introduction to Graph Databases

Register for East Anglia at Eventbrite

Friday 23/04/2021 – Dumnonia

Annette Allen  Developing code to keep your DBA happy

Rob Sewell  Introduction to Azure Arc enabled Data Services

Register for Dumnonia at Eventbrite

Register to start your epic voyage!

The SQL Saturday Foundation

After all of the uncertainty in 2020, 2021 has brought about the exciting reboot of SQL Saturday. SQL Saturday has always been there throughout my career and provided an amazing place for learning and networking.

Since the end of PASS last year Steve Jones, Voice of the DBA, has been working on Moving the Community Forward. The vision is to continue to facilitate and promote local, free events organized by individuals in some area. 

This revival has been moving forward at great speed towards creating a foundation and building a future for SQL Saturday.

There are a number of blogs where you can find out more about the SQL Saturday vision.

The SQL Saturday Foundation Vision

Rebuilding SQL Saturday–Picking a Board of Directors

Finding a SQL Saturday Board of Directors

I am amazingly excited and honoured to be in the first set of directors. The initial directors are: 

• Andy Warren  (b | t) – USA
• Carlos Lopez (b | t) – LATAM
• Heidi Hastings (t) – APAC
• Johan Brattås (t) – EUR
• Stacia Varga (b | t) – USA
• Steph Locke (b | t) – EUR
• Steve Jones (b | t) – USA
• Taoib Ali (b | t) – APAC/USA
• Dr. Victoria Holt (b | t) – EUR

Moving forward through 2021 I hope everyone is encouraged to create a SQL Saturday to help everyone keep learning and connected. There are some great suggestions in the post on how to do this with ease.  

Everyone run a SQL Saturday in 2021

Developing post-pandemic data strategy

The ODI have a really interesting podcast on data strategy . It is the  fourth episode of the ODI Inside Business podcast, with ODI’s Learning and Business Development Director, Stuart Coleman and Robin Sutara, newly appointed Chief Data Officer at Microsoft UK, to discuss why culture change should power organisations’ data strategy plans.  They discussed what cultural and mindset shifts are needed to facilitate the implementation and execution of a viable data strategy.  As well as how Microsoft has shifted their thinking in their journey and how they have succeeded in helping their clients transition into a new digital thinking era.

The podcast is entitled "Inside Business – Developing post-pandemic data strategy: Why culture change should power your plans"

There were three important areas that were raised when thinking about data strategy. 

1. Data Capabilities - where a business / department is on their data journey. Are they modernising their legacy systems, thinking about data science or data governance?

2. Data Culture - how do you instantiate a data culture in an organisation? This can be by democratizing data ensuring the right data is defined at the right time and also ensuring employees feel empowered to act on the data without looking to senior leadership

3. Encouraging a growth mind set, fail fast, iterate, make decisions that will drive progress and to iterate as the data changes.

Data as a strategic asset

The Chief Data Officer (CDO)  role is an interesting new role which varies widely between organizations. It can cover a variety of responsibilities, from driving data technology decisions, to creating data-driven cultures, data governance and identifying data-centric opportunities.

The ODI has set out a vision in Data strategy: how an ecosystem approach can help shape your vision . Also the ODI Inside Business – a checklist for leaders article contains some useful points. They mentioned appointing a Chief Data Officer is a good place to start. This role is a leader and should be considering the company’s data skills, data handling and data infrastructure as part of its responsibilities.

Their checklist suggests five critical areas leaders can explore through a series of questions.  There is a list of tools to use to take practical action.

• Data strategy: How data is used differently in your organisation to create value and improve business performance, in the face of competition.
• Data infrastructure:  The datasets, policies, systems, processes and tools you need for data to create value.
• Data ecosystems: The internal and external networks that enable data to be accessed, used and shared.
• Data skills: The literacy and skills your people need to improve business performance.
• Data ethics:  How your business creates value from data whilst avoiding harmful impacts.

It is also worth reading the book Growth Mindsets by Carol Dweck. In the book she discusses that there are two mind sets: 

• A fixed mindset - intelligence is static
• A growth mindset - intelligence can be developed

A growth mindset drives success. Thus having a growth mind set and looking at the five critical areas will help define the data ecosystem of the future.

Responsible AI

Responsible AI is an important topic to research and to see how to adopt this approach.

Microsoft lists its six key principles for responsible AI: accountability, inclusiveness, reliability and safety, fairness, transparency, and privacy and security. Creating responsible and trustworthy AI is most important and these are essential principals which have two perspectives: ethical and explainable. The Microsoft AI Principles for AI systems are 

• Fairness - should treat all people fairly
• Reliability & Safety -  should perform reliably and safely
• Privacy & Security - should be secure and respect privacy
• Inclusiveness - should empower everyone and engage people
• Transparency - should be understandable
• Accountability - People should be accountable for AI systems

Data Skills Framework

The ODI have created a useful methodology for looking at data skills. It classifies the data landscape skills into technical skills, and other skills such as service design, data innovation and change leadership which are important for organisations to succeed.  I would recommend looking at the resource page as data literacy is most important to address and this framework helps look holistically at data skilling. 

Data Toboggan - Slide Preparation Inaugeral event today!

Join Data Toboggan -Slide Preparation for our first Azure Synapse MeetUp on Thursday, March 25, 2021.

For this edition, Andy Cutler (@MrAndyCutler) will be presenting "Building a Data Warehouse Dimensional Model using Azure Synapse Analytics SQL Serverless" for your enjoyment.

A summary of his talk is below

"In this session we'll be looking at using Azure Synapse Analytics SQL Serverless to create a Data Warehouse using the Dimensional Modelling technique. All data will be stored in an Azure Data Lake Gen2 account with processing and serving performed by the SQL Serverless engine."

Sign up here

Data Virtual Conference Extravaganza

The whole last year has certainly been one we will all remember as the year everything changed. Not defeated by the pandemic many new virtual conferences have sprung to life to help with continuous improvement and learning.

Data Toboggan

A specialise conference on Azure Synapse. The slope that enables predictive analytics. It runs 3 conferences a year which are 12 hour long events.

New Stars of Data

The launchpad for new speakers in the Microsoft Data Platform space

New Stars of Data is an event to help find and promote new speakers in the Microsoft Data Platform arena. It is run by Ben Weissman and William Durkin. Speakers submit their presentation ideas and are assigned to a seasoned speaker, who will mentor the newcomer in all aspects around delivering a presentation.

Data Weekender

A Virtual Popup Microsoft Data Conference. #DataWeekender is a new online conference that first ran on Saturday 2nd May 2020. Our original intention was to put on an online data platform conference for the community to fill that void left by the canceling of the in-person events due to Covid-19.

EightKbConf 

A virtual MS Data Platform conference focusing on SQL Server internals.

DataMinutes

The fastest conference for the Microsoft Data Platform Community. Lightning talks only. Organized by the @datagrillen guys @sql_williamd and @bweissman

Geeks on Screens with Coffee

Episode 67 - Rushing through the snow

I had the pleasure of chatting with Mark Pryce Maher on Geeks on Screens with Coffee on Thursday 11th March 2021 at 6:15pm (GMT) .

I joined the call with my SQLBits Speakeasy mug which came from the Manchester Central event, 27 February to 2 March 2019 and we talked about many things including the creation of Data Toboggan, Data Relay and Music. The video is here.

International Women's Day 2021

It’s international women’s day (IWD) again on 8^th March. Time to celebrate accelerating gender parity. It important to celebrate women’s achievements and call out inequality.  Making things more visible is important.

Celebrate women in tech and innovation . I am pleased to be a part of the Million Women in STEM project.  A list of the biggest, most diverse, global, women in stem campaign & community. Their aim is to spotlight women in STEM, to provide visible role models though their campaign, community, and outreach to inspire the next generation of girls.

This year’s IWD theme is #ChooseTo Challenge

A challenged world is an alert world and from challenge comes change.
So let's all choose to challenge.

How will you help forge a gender equal world?
Celebrate women's achievement. Raise awareness against bias. Take action for equality.

I am pleased to be a part of the Inclusion and Diversity team at Coeo with a day of celebration. Read our blog. 

Microsoft Ignite March 2021

An amazing keynote as ever. Not so many data announcements this time but that is not to say it wasn’t exciting. Satya Nadella mentioned these 5 key attributes to drive innovation forward utilizing cloud technologies for tech intensity. 

1. Ubiquitous And Decentralized Computing

With the radical change in computing architecture and systems from cloud to edge it will result in exponential growth in compute capacity. This transforms interactions between people places and things. With physical and digital worlds converging there is a need for more sovereignty and decentralized control

2. Sovereign data and ambient intelligence

In this world, data will be more private, more sovereign. Data governance and providence will take on new importance. There will be new methods of federated machine learning to drive the next generation of personalized and yet privacy-preserving services

3. Empowered Creators And Communities Everywhere

There needs to be more skills, tools and platforms and collaboration across communities to enable creation.  It may be building a virtual world, formulas in spreadsheets or low code tools to build application.  The democratization of creation will accelerate and drive new innovation

4. Expanded Economic Opportunity For The Global Workforce

There needs to be flexible working to enable productivity for inclusive collaboration, learning and wellbeing with continuous feedback.

5. Trust By Design

There needs to be ethical principles govern the design, development and deployment of AI. The technology needs to be secure by design and build technology with the design intent to protect the fundamental rights of all people

Microsoft Mesh

Microsoft Mesh, a new mixed-reality platform powered by Azure that allows people in different physical locations to join collaborative and shared holographic experiences on many kinds of devices. It makes me think science fiction just became reality.  This is a major shift in maturity and the art of the possible for connecting from anywhere.

Data announcements 

Harness the power of data with Azure Data and AI

New enhancements for Azure Cosmos DB, Azure Purview, and Azure Synapse.

Azure Percept

With Azure Percept, Microsoft adds new ways for customers to bring AI to the edge

Azure Synapse Pathway

Announcing Azure Synapse Pathway: Turbocharge your data warehouse migration

Then for Power BI Some very big news. PowerBI Premium Per User (PPU) pricing is announced.

The #MSIgnite Book of News covers all the announcements. 

Data Toboggan - Slide Preparation

We have been working behind the scenes and are so excited to launch Data Toboggan - Slide Preparation our usergroup arm.

We're working on some great content, but as this is our first one we wanted to test the water, so our first evening session will be just 1 hour (speaker* permitting - overruns are not uncommon in our line of fun), and we hope you'll join us at the top of what we're hoping will be a great run :-)

Later editions will vary in length and start time. Hey, we're new at this and don't know what works yet. It'll be top class, though. Promise.

We are cloud born and welcome local, national and international speakers and attendees.

The first Azure Synapse virtual meetup is Thursday March 25, 2021 5:00 PM GMT http://bit.ly/DT-SPMeetup

Data Toboggan-Cool Runnings

Don't forget our next main 12 hour summer edition conference is on 12 June 2021.

Call For Speakers is Open. Submit now http://bit.ly/DT-CRCFP Register Now http://bit.ly/DT-CRRegister

We hope you have been enjoying the session videos from the January 2021 Data Toboggan event http://bit.ly/DataTobYouTube

UK Research and Data Research Centres

I have been watching the research industry grow in the UK. Having a research industry is really important to drive innovation. There is a UK research development roadmap.  UK Research and Innovation (UKRI) is a key enabler for Research and Development centers. 

Above are the UK government’s Public Sector Research Establishments and UK Research and Innovation-funded Institutes. 

The UK funding landscape

There are many other places where data, machine learning, and AI  are a core part of research and innovation centres. Here are a few UK based places.

UK Health Data Research UK

UK Research Data Discovery Service

Consumer Data Research Centre

National Innovation Centre for Data

The National Institute for Data Science and Artificial Intelligence

Leverhulme Centre for the Future of Intelligence

UK Data Archive

The Institute for Advanced Automotive Propulsion Systems

National Quantum Computing Centre

Centre for Data Ethics and Innovation

The Open Data Institute

Azure Data Community Announcement

A special announcement was made by Buck Woody on Data Exposed. Microsoft expands on its announcement  last year following the collapse of PASS that is committed to the Data Community and say local users groups are the life blood of the community. The announcement is here 

Assets have been created that are Community-owned and Microsoft-empowered. These are resources and benefits to enable local community leaders to run their groups. 

Assets, Resources and Benefits 

Azure Data Community Gateway

It is a gateway not a destination. This resource is for community events, finding MVPs in the Azure data area and other resources such as diversity and inclusion that the community may find interesting.

Access to Meetup Pro

For qualified community groups to schedule, track events and for implementations of our local events

Microsoft Teams for Community Groups

Providing a full community Teams edition to qualifying community group leaders for chat, video, voice, file storage and applications. It comes with a predefined community template. It also allows you to connect with other community groups.

Community Leader Collaboration

Also Microsoft will be facilitating a periodic meeting of community group leaders to enable sharing of practices.

Resources for Self-Directed Local Groups

This is for established groups.

These resources and tools are for technologies focusing on

Effective community groups make it local, make it helpful for local jobs and improving your career and being champions for diversity and inclusion. Effective community groups are impactful.

The url is aka.ms/datacommunity

In summary this is an amazing resource for local community user groups to thrive, expand and encompass diversity and inclusion.

Azure Data Community FAQs

For answers to your questions 

PASS Data Community

Redgate has been amazing taking over the data community flag of PASS. They are sharing resources at speed with the community.

PASS Summits

Redgate commits to the future of PASS

They feel in-person events aren’t going to be an option for a while, so they are thinking about planning the Summit as a virtual community conference again this year.

“PASS Pro” Educational series 

Redgate announce that the expert-led educational training courses covering the following four topics are available to all: Data Modernization, Azure Migration, Azure Synapse, Azure SQL. Watch the sessions and learn from a range of experts online here

PASS Summit 2019 Learning Pathway 

These videos are arriving on PASStv on Youtube

Data Toboggan-Cool Runnings

We are really excited to announce our summer edition of  Data Toboggan-Cool Runnings on 12 June 2021. 

The summer edition is named after Cool Runnings,  a 1993 American comedy sports film. A Jamaican bobsleigh team fights many odds, including a reluctant coach, to compete in the 1988 Winter Olympics. 

The phrase 'cool runnings' is defined in the film to mean: peace be the journey.  As well as it being an enjoyable film it contains some life lessons

• Believe in yourself. The underdog can win 
• Create your own way forward
• Have a positive self image
• Being yourself is important
• Have conviction to follow your own path
• Don't let winning change the person you are
• Everyone deserves a second chance
• Have perseverance to reach the end

Our summer edition of Data Toboggan-Cool Runnings is on 12 June 2021 Call For Speakers is Open http://bit.ly/DT-CRCFP Register Now http://bit.ly/DT-CRRegister #AzureSynapse @Azure_Synapse

In our Call for Speakers we are accepting two types of sessions;

1) 45 min - live talk on an aspect of Azure Synapse.

2) Upto 5 min pre-recorded talk. We will play these in between the sessions. These could be on anything Synapse related, tips & tricks, etc.

Submit your sessions and or register for the event. We would love to see you there. 

Creating a Conference

What a few weeks this has been. I have had an amazing amount of fun working on this new conference idea. A specialist conference. I have been amazingly lucky that two of my friends, Mark and Richard, have been so kind to help me bring this conference to light. From creating the logo, excellently designed by Nightingaleai, to the design of the the structure of the day, the speaker liaison, to registration and of course the platform presentation layer, there was a lot to be done.  

Creating a conference from scratch in 2 months with 2 weeks off for Christmas was rather ambitious. I had an idea about the brand due to my love of snow and having fun in the snow. I wanted the conference to be fun, sharing and inclusive just like making snowmen, sledging, skiing and ice skating. These are all best done in a group. I had a chat with one of my friends and the conference was born.

These are things we did

We didn't know how well the conference would be received. Whether a specialist conference was the right way to go or not. We were completely blown away by the response by the speakers agreeing to give their time to share their knowledge and the attendees. From the stats we have the attendees stayed with us most of the 12 hours. Some got up at 2.30am to start watching, some stayed up into the early hours of the morning.  To you who attended we thank you for attending. 

A few of the comments which made our day for our first event

"Brilliant all day today; content, speakers and for the most of it, the technology."  

"Looking forward to the next one"

"It was nearly like attending a SQL Saturday or Data Relay event. Without the coffee & biscuits, and hotel bar."

"excellent conference. Thank you so very much for all your work in this. Intro sessions were well pitched but I got loads out of the more in depth sessions."

"The quality of all the sessions was astonishingly high - I was amazed during the whole day... And huge congrats to the organizers!"

We had a dedicated chat channel for attendees to have a general chat, ask speakers question. We ran with 45 minuet sessions  starting on the hour with built in gaps for overruns, Q& A, Coffee and there was space between the sessions for Mark and his amazingly fun puppets.  

I think an element of fun is what is required.

We also had lightning sessions to enable new talent and did manage diversity in our speakers and organizing team.

So for a few of high level statistics

Last of all we do have a conference survey to complete. It is a very short 6 questions. Thank you again from all of us for your support.

Data Toboggan Agenda

The Data Toboggan Agenda for Saturday 30 January 2021 is

7:45 AM Keynote

8:00 AM An Introduction to Azure Synapse - What is it? Why use it? And how? - Paul Andrew

9:00 AM Developing HTAP Analytical Solutions with Azure Cosmos DB and Azure Synapse Analytics - Will Velida

10:00 AM Pipelines and Data Flows: Introduction to Data Integration in Azure Synapse Analytics - Cathrine Wilhelmsen

11:00 AM Machine Learning in Azure Synapse - Terry McCann

11:45 AM Lightning Sessions: Building a Data Warehouse Dimensional Model using Azure Synapse Analytics SQL Serverless - Andy Cutler

12:00 PM Synapse SQL Serverless Performance Characteristics - Luke Moloney

1:00 PM Spark Showdown - Databricks VS Synapse Analytics - Simon Whiteley

2:00 PM Automated Machine Learning (AutoML) in Azure Synapse Analytics - Ferenc Csonka

2:45 PM Lightning Sessions: Harnessing Azure Synapse Analytics SQL Serverless in Power BI Dataflows - Andy Cutler

3:00 PM Practical Data Distribution in Dedicated SQL Pools - André kamman

4:00 PM Azure Purview Microsoft's answer to Data Governance and Data Lineage - Erwin de Kreuk

5:00 PM Back the Truck Up: Data Loading Design Patterns for Azure Synapse Analytics - Jason Horner

6:00 PM Azure DevOps duet - Data Toboggan edition - Kevin Chant & Sander Stad

7:00 PM AI powered Analytics using Azure Synapse Analytics - Anupama Natarajan

Data Toboggan 12 hours of Azure Synapse

Data Toboggan are running a 12 hours of Azure Synapse conference touching on almost every aspect of it, Workspace Dedicated Pools, Serverless Pools, Spark, Integrate Synapse Link and more.   

Register now : 30 Jan 2021 (8am-8pm GMT) http://bit.ly/datatobregister 

The Agenda:  http://bit.ly/datatobagenda

Read the blog Get to grips with Azure Synapse with this free digital event to find out more about the event.

Polaris The Distributed SQL Engine in Azure Synapse

 Very Large Data Base Endowment Inc. (VLDB Endowment) is a non-profit organisation incorporated in the United States for the sole purpose of promoting and exchanging scholarly work in databases and related fields throughout the world. The VLDB conference, usually annually, is an academic conference focusing on data management technology.

The academic papers that they publish are all available online for free. One such paper describes the SQL query engine, Polaris, in Azure Synapse. The academic paper can be read here.

Josep Aguilar-Saborit, Raghu Ramakrishnan, Krish Srinivasan, Kevin Bocksrocker, Ioannis Alagiannis, Mahadevan Sankara, Moe Shafiei, Jose Blakeley, Girish Dasarathy, Sumeet Dash, Lazar Davidovic, Maja Damjanic, Slobodan Djunic, Nemanja Djurkic, Charles Feddersen, Cesar Galindo-Legaria, Alan Halverson, Milana Kovacevic, Nikola Kicovic, Goran Lukic, Djordje Maksimovic, Ana Manic, Nikola Markovic, Bosko Mihic, Ugljesa Milic, Marko Milojevic, Tapas Nayak, Milan Potocnik, Milos Radic, Bozidar Radivojevic, Srikumar Rangarajan, Milan Ruzic, Milan Simic, Marko Sosic, Igor Stanko, Maja Stikic, Sasa Stanojkov, Vukasin Stefanovic, Milos Sukovic, Aleksandar Tomic, Dragan Tomic, Steve Toscano, Djordje Trifunovic, Veljko Vasic, Tomer Verona, Aleksandar Vujic, Nikola Vujic, Marko Vukovic, and Marko Zivanovic. 2020. POLARIS: the distributed SQL engine in azure synapse. Proc. VLDB Endow. 13, 12 (August 2020), 3204–3216. DOI:https://doi.org/10.14778/3415478.3415545 

The paper shares information on the new serverless SQL Pools service within Azure Synapse. It is an interesting read. Simon Whiteley discusses this paper which can be seen on You Tube. 

2021 The year of hope and change

Happy New Year. It is 2021 already with a sense of hope.  The dawn of a new year brings excitement of what lays ahead and what changes we will bring into our lives. The world has changed in incomprehensible ways and i'm not sure it will ever go back to the way it was. A sense of family and local community connectedness has grown throughout last year and I hope this is a positive out of the dark days. A great sense that ripples around us is that protecting the environment should be a core of our everyday lives.  

Whilst we have a difficult few months ahead there is light at the end of the tunnel. 

I watched an inspirational film over Christmas Hidden Figures (2016). I hadn't seen it before. It was about an African American female mathematicians who worked at the National Aeronautics and Space Administration (NASA) during the Space Race from a book by Margot Lee Shetterly. I must read the book. 

I wish everyone a safe, healthy and prosperous new year.