Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Friday 20 August 2021

Maximize the value of your data with Azure Purview

There is a digital event coming on 28 September 2021 9-10am pacific time. The event is about achieving unified data governance with Azure Purview.
Join Microsoft Corporate Vice President Rohan Kumar at this free digital event for demos and deep dives.
Register now to:
  • Learn to create a comprehensive, automated map of all your data.
  • See how Azure Purview works with Azure Synapse Analytics, Power BI, and the rest of your data estate to deliver timely, reliable insights.
  • Watch in-depth demos of product features including Azure Purview Data Map and Data Catalog.
  • Ask Azure experts your data governance questions in the live Q&A.


 

Thursday 19 August 2021

Azure Purview August updates

There have been some exciting changes to Azure Purview announced. These changes relate to charging and permissions. 

Elastic data maps

The data map is the foundation for data discovery. The data map has two components the throughput created by CRUD operations and storage of the metadata.

The data map can now grow elastically starting at one capacity unit. A capacity unit includes a throughput of 25 operations/sec and the metadata storage scales in increments of 2 GB. Purview Data Map can automatically scale up and down within limits . This new charging model makes Purview much more user friendly and less costly to set up and run.









Access Control in Azure Purview

Note: only applies for Purview accounts created on or after 18 August 18 2021,

A collection is a tool to group assets, sources, and other artifacts into a hierarchy for discoverability and to manage access control. Collections are used to organise and manage assets.

The are various roles that exist:

  • Collection admins - can edit Purview collections, their details, and add sub collections. They can also add users into other Purview roles on collections where they're admins.
  • Data source admins - can manage data sources and data scans.
  • Data curators - can create, read, modify, and delete catalog data assets and set up relationships between assets.
  • Data readers - can access but not modify the data.













There is a great video to watch from W Strasser explaining this new data plane RBAC (role based access control) catalog permission. Being able to fine tune the access to the collections brings with it great advantages. Currently collection names can't be updated or deleted. 

Monday 9 August 2021

A Summer Retrospective: a bygone era


A few weeks in rural France is just the place to contemplate life and take a step back into a bygone era. An era where there are no phones, no internet and no television. Life can quite easily pass you by and you could go for weeks not speaking to a sole. The fruit on the trees ripen in the orchard, the birds waiting for the perfect moment to swoop and eat the fruit. The roads are mostly empty with the occasional car or logging lorry passing by. Cycling is heaven with the roads to yourself.

This rural area, 214 million years ago, had all life within 300 miles of Rochechouart wiped out when a meteorite, around one of the 15 largest ever to come crashing down on earth. The geological signs of he creator are still present today. This bygone era was also rife with conflict. From the last battle of Richard the 1st - the Lionheart, who laid siege to the Chateau of Chalus-Chabrol, located at the border between Aquitaine and the French kingdom, to the hideouts in the forest of the Maquis du Limousin, who were one of the largest groups of French resistance fighters in the Second World War. The village of Oradour-sur-Glane remains an empty ruin as a memorial for the massacre of its inhabitants.

In this backdrop, technology seems a lifetime away. I can't stress enough the tremendous benefits of taking a technological break for your mental health. You can dream and innovative without the interruption of everyday life.

The age of cloud computing, big data and the algorithm requires a 360-degree perspective. A socio-technical perspective is critical. Reflecting on the changes to the earth, made to this unique landscape from space, you realize that data is in the environment. It is not possible to be an expert in all areas as data is the environment. Data is history, is in the maps, is used in conflict resolution and is used for impact analysis. Data is completely inseparable from life and it drives life, not only business. The  choice of tools available help you navigate through data are vast.

The question is can one truly ever master the entirety of life. Data is life, the past, the present and the future. To truly be a master it requires collaboration, communication and control, as data weaves its interconnected complexity throughout life. A holistic view of this diverse scientific area is required to provide a sustainable future. There is no one best practice that can help navigate this web of graph vertices and edges.

To that end I summise that taking a technological break enables the mind to contemplate and blue sky thinking roam free. Happy Summer break. 

Thursday 5 August 2021

Data Conferences


I have been surprised by the number of data conferences there are and the continuing growth and diversity of the topics and formats. As well as the main and specialist conferences, a few of which are listed below,  there are a huge number of training events that take place in the evenings and weekends such as  SQL Saturdays ,  Azure Data Community and Data Saturdays . With all these events providing learning opportunities ,and many for free, we are very lucky that we have a community that is so will to share their experience.

Conferences

Date

URL

Dativerse

13-Aug-21

https://datagrillen.com/dativerse/

Data Platform Virtual Summit 

13-18 Sept 2021

https://dataplatformgeeks.com/dps2021/

Future Data Driven 

29-Sep-21

https://datadrivencommunity.com/

DataMinds Connect  

11-12 Oct 2021

https://datamindsconnect.be/

New Stars of Data  

22-Oct-2021

https://www.newstarsofdata.com/

Data Weekender 

06-Nov-2021

https://www.dataweekender.com/

PASS Data Community Summit

8-12 Nov 2021

https://passdatacommunitysummit.com/

The SQL Server & Azure SQL Conference 

7-9 Dec 2021

https://www.mssqlconf.com/#!/

Data & AI Summit 

27-30 June 2022

https://databricks.com/dataaisummit

Big Data LDN

22-23 Sept 2021

https://bigdataldn.com/

SQLBits 

2022

https://sqlbits.com/

Azure Cosmos DB Conf

20-21 April 2021

https://gotcosmos.com/conf

Power BI Summit

7-11 March 2022

https://globalpowerbisummit.com/

Data Toboggan

12-Jun-2021

http://www.datatoboggan.co.uk/

Microsoft Build 2021

25-27 May 2021

https://mybuild.microsoft.com/home

Microsoft Ignite 2021

2-4 Nov 2021

https://myignite.microsoft.com/home

Microsoft Inspire 2021

14-Jul-2021

https://myinspire.microsoft.com/home

DataMinutes

22-Jan-2022

https://datagrillen.com/dataminutes/


Monday 2 August 2021

Responsible Innovation: A Best Practices Toolkit

Responsible innovation is a toolkit that helps developers become good stewards for the future of science and its effect on society.  

There are 3 areas
  • Judgment Call
  • Harms Modelling
  • Community Jury

This toolkit provides a set of practices currently in development, for anticipating and addressing the potential negative impacts of technology on people. This is an early release of this development.

Judgment Call 

Judgment Call is an award-winning game and team-based activity that puts Microsoft’s AI principles of fairness, privacy and security, reliability and safety, transparency, inclusion, and accountability into action. The game cultivates stakeholder empathy through scenario-imagining. Game participants write product reviews from the perspective of a particular stakeholder, describing what kind of impact and harms the technology could produce from their point of view.

To prepare for this game, download the printable Judgment Call game kit.























Harms Modelling 

Harms Modelling is a framework for product teams, grounded in four core pillars of responsible innovation, that examine how people's lives can be negatively impacted by technology: injuries, denial of consequential services, infringement on human rights, and erosion of democratic & societal structures. Similar to Security Threat Modelling, This modelling enables product teams to anticipate potential real-world impacts of technology.



Community Jury

Community Jury is a technique that brings together diverse stakeholders impacted by a technology. It is an adaptation of the citizen jury. The stakeholders are provided an opportunity to learn from experts about a project, deliberate together, and give feedback on use cases and product design. This responsible innovation technique allows project teams to collaborate with researchers to identify stakeholder values, and understand the perceptions and concerns of impacted stakeholders.

These 3  new tools under development are underdevelopment but quiet interesting to look at. 

References

Citizens Juries

The Ethics of AI Ethics: An Evaluation of Guidelines

Hagendorff, T. The Ethics of AI Ethics: An Evaluation of Guidelines. Minds & Machines 30, 99–120 (2020)