Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Thursday, 2 July 2020

Microsoft MVP for a third year

During such a life changing time, I am over the moon with joy to be honoured with my third Microsoft Most Valuable Professional (MVP) award in recognition of exceptional technical community leadership for Data Platform for 2020-2021. How amazing to receive this. Thank you @MVPAward #MVPBuzz . There are so many amazing MVP's who help the community improve and grow their data platform knowledge. I am passionate about helping share the knowledge and experience I have gained.

This has been a strange period with many events cancelled and turned into virtual events. As an organizer of  Data Relay this year we cancelled our event to make way for SQLBits to move into the only viable slot this year. As things are still ongoing I think it is unlikely any in person conferences will continue this year.  However there are some amazing free online events to attend. My thoughts have been on helping and supporting the local community as well as the data community. 

Friday, 19 June 2020

Introduction of Power BI

More and more people are using Power BI in their everyday roles. Understanding the different deployment types and basics of the product helps with the adoption of self service data and digital migration. Not every organization has the appetite or capability to use Power BI in the cloud.  The cloud and on premises capabilities do have some differences but it is a great place to engage with dashboards and paginated reports.  The presentations I have given covered these areas.  






Tuesday, 9 June 2020

STEM Role Models - 1 Million Women In STEM

This is women in STEM. They are celebrating 1000 real role models that are kicking ass,smashing stereotypes & breaking barriers. This is an amazing chance to help share women STEM role models for the future generations. Read more and join us 1mwis.com


#womenintech #womeninmaths #WomenInScience #womeninengineering #AcademicTwitter #WomenWhoCode

Wednesday, 3 June 2020

Planning a PowerBI Enterprise Deployment

There is an updated version of the Planning a Power BI Enterprise Deployment whitepaper. Deploying Power BI in a large enterprise is a complex task that requires a lot of thought and planning.

The paper includes these areas for consideration.

Section 1: Introduction Section 2: Power BI Usage Scenarios Section 3: Power BI Architectural Choices Section 4: Power BI Licensing and User Management Section 5: Power BI Source Data Considerations Section 6: Power BI Dataset Storage Options Section 7: Power BI Data Refresh and Data Gateway Section 8: Power BI Dataset and Report Development Considerations Section 9: Power BI Collaboration, Sharing and Distribution Section 10: Power BI Administration Section 11: Power BI Security and Data Protection Section 12: Power BI Deprecated Items Section 13: Support, Learning, and Third-Party Tools

There is a summary of some of the changes at coatesdatastrategies.com/blog/updated-w

Monday, 1 June 2020

Ethical Data Handling Strategy


As a part of the Ethical Data Handling Strategy in my Data Quality Framework there are various things to consider. A good place to start is with The Data Ethics Canvas which can help you identify and manage ethical data issues. 


ODI have produced a clear definition of what data ethics is

“Data ethics is a branch of ethics that evaluates data practices with the potential to adversely impact on people and society – in data collection, sharing and use” The Open Data Institute, 2018.


Data Ethics is important to consider and embed in your data system now. With the use of data in AI expanding it is important to obtain create an Ethical Data Handling Strategy.

Monday, 25 May 2020

The Future of Tech

I enjoyed watching this The Future of Tech session, with Kevin Scott and guests at Microsoft Build. The session discusses advances in large scale models for natural language generation and AI on the intelligent edge among other things. Watch it here


Saturday, 23 May 2020

Build Book of News 2020

What an amazing inspiring conference, where it is possible to make dreams become reality. The world is changing, reimagine tomorrow.

The Book of News 2020 shares some amazing advances to help shape the world to come.

The Microsoft Build 2020 Book of News is the guide to the key news items that were announced at Build.


Thursday, 21 May 2020

Sketch the docs

Great to see an interesting technique shared about visual storytelling. A summary about Sketchnoting and Zines is in the last 15 minutes. There were other sessions are Build

Video: https://aka.ms/msbuild2020-sketchnoting-video
Slides: https://aka.ms/msbuild-sketchnoting-slides
Site: http://sketchthedocs.dev
Blog: http://dev.to/nitya






















Wednesday, 20 May 2020

2020 Build Keynote

The Build Keynote from Satya Nadella was entitled empowering every developer,  on Tuesday 19 May. You can watch it here.

In this time of uncertainty, developers will play a central role in reimaging the world we live in and accelerating our path to recovery. The 3 phases he touched on emergency, recovery and reimaging. Going forward business will need to be able to remote everything at a moments notice, automate everywhere to be agile and simulate anything. The Power Platform, Azure Arc as the first control plane and Teams are enabling the future. Satya left us with a thought provoking statement.

"We are at an inflection point. As developers you have that opportunity, as well as a responsibility, to define what should be rebuilt, what should be reimaged, and what should be left behind." Satya Nadella

We are crossing into a new frontier, anywhere together.



A few of the many interesting announcements follow: 

Microsoft responsible machine learning capabilities build trust in AI systems, developers say
Build AI you can trust with responsible ML

Autoscale is now generally available on AzureCosmosDB, and the launch the public preview of their new serverless model in just a couple of months!


Microsoft Build brings announcements for cloud data, analytics services, and intersection of the two
https://www.zdnet.com/article/microsoft-build-brings-announcements-for-cloud-data-analytics-services-and-intersection-of-the-two/
Azure SQL Edge now in preview
https://azure.microsoft.com/en-gb/updates/azure-sql-edge-now-in-preview/

Microsoft announces a new supercomputer and lays out vision for future AI work.
https://blogs.microsoft.com/ai/openai-azure-supercomputer/
It has built one of the top five publicly disclosed supercomputers in the world, making new infrastructure available in Azure to train extremely large artificial intelligence models.



Thursday, 14 May 2020

Microsoft Build is coming to your home

Sign up to watch Microsoft Build, a digital event, May 19-20 2020. Microsoft Build delivers a new way to create unique opportunities for us to explore the latest developer trends and tools in a series of detailed, expert-led key segments. This is always an amazing conference that empowers the developer.


Friday, 8 May 2020

Microsoft Business Applications Summit 2020 Excitement

Microsoft Business Applications Summit took place online this year 6-7 May 2020 for free. New features were shared and the sessions discussed how to optimize operational processes. I was particularly interested in Power BI. There was an emphasis on Data Culture for every decision. The new features are intelligent and familiar, meet the needs of business and IT, in a unified way, and are ubiquitous, for everyone to have access. Trust is the foundation with BI for collaborative teamwork and BI being woven into the fabric of the organization.






















There are some amazing AI features with the natural language smart narratives, being able to suggest questions in Q and A and there is trend analysis automatically generated over points.





















The data stack for Power BI collects query patterns and determines materialized views for optional performance. Microsoft Information Protection in Power BI is GA. For admin and governance you can bring you own log analytics. Deployment pipelines are coming as part of the application lifecycle management.

Lineage view and dataset impact analysis are two useful additions to the toolset.  Lineage view recently became generally available. Lineage view enables you to visualize the entire data journey from a given workspace, making it easy to determine the upstream and downstream dependencies of artifacts, examine artifact information, and take action. Dataset impact analysis enables a dataset owner to understand how a dataset is being used within the workspace and in other workspaces. With the dataset impact analysis, it is possible to determine if a dataset is being used in mulitple workspaces, so the impact of changes can be reduced.



There will be templates to help people start with Power BI like other Microsoft products. There were so many other new feature announced. You can watch the sessions from the day on-demand.


Saturday, 2 May 2020

Data Quality Framework

Data quality is defined as the planning, implementation, and control of activities that apply quality management techniques to data, in order to assure it is fit for consumption and meets the needs of data consumers. (The Data Management Body of Knowledge)

The trustworthiness and quality of our data has real impact in the world and on society influencing critical decisions. To help ensure the data we collect from around us helps answers questions and provide informed answers, a data quality framework is required. There are many components required to help improve data quality.  The Data Quality dimensions that can be measured to manage improvement are:

  • Accuracy
  • Completeness
  • Consistency
  • Integrity
  • Reasonability
  • Timeliness
  • Uniqueness
  • Validity
  • Accessibility

Below is my data quality framework.



Thursday, 30 April 2020

Rainbows of the new world

I have been working at home since mid March, living in a world that doesn't seem real. The pandemic has changed the world we live in currently. We have homes full of family, or none at all, we shop for those needing to be shielded and those selfless brave NHS staff, carers, shop worker, rubbish collectors and all, still work everyday to help the people with essential services.

This is a new age where an opportunity awaits for us to improve our lives. Many of us have been enforced into a social experiment which has shown working from home is possible with new tools that exist. Also saving the planet became possible without all the car emissions created by people travelling to work and delivering children to school by car. Our local communities became more connected and peoples started walking round their local areas.

Putting the planet and our health first is crucial to our survival. The need for urban parks and gardens has increased within cities for people mental health. With the windows full of rainbows, it has given hope and activity. Open air trim trails, local heritage trails and outside museums could help keep learning fresh.

We are seeing the need for a connected world by default and the major upgrade of internet services enables a rapid transition to the cloud services for business and home life. Cyber protection is required more than ever in this diverse world. Self driving cars and robots have the potential to increase deliveries to the door. Investment in health and AI for health has never been higher.





Wednesday, 29 April 2020

Microsoft Ignite 2020

Microsoft Ignite will be back in September 2020. Microsoft Ignite will be launched as a complimentary digital event experience this September. Join them to learn innovative ways to build solutions, migrate and manage our infrastructure, and connect with Microsoft experts and other technology professionals from around the globe.

It is also possible to view all sessions from last year’s Microsoft Ignite.


Monday, 27 April 2020

Data Weekender

A Virtual Popup Microsoft Data Conference is happening on Saturday 2nd May 2020 https://www.dataweekender.com/

Data Community Weekender Europe is a new online conference hosted by various professionals from around Europe. With many events cancelled or postponed across Europe (and the world) in March, April and May, they wanted to put on an online data platform conference for the community to fill that void left by the cancelling of the in-person events on a day that hopefully, most people can attend.


This is set to be an amazing event.



Thursday, 23 April 2020

Microsoft Business Applications Summit

The Microsoft Business Applications Summit is now a digital event on May 6, 2020. Microsoft say get ready for the new digital event experience, with opportunities to hear from Microsoft executives, learn from experts, and connect with your community. Register today, at no cost to you!

The event will let you learn about the latest innovations and what is coming next in tools. Discover how Microsoft business applications work together to unify data to gain in-the-moment intelligence. In addition learn and explore the new capabilities across the Microsoft Power Platform.




Thursday, 9 April 2020

Spark AI Summit 2020

The largest data and machine learning conference, Spark and AI Summit brings together over 7,500 engineers, scientists, developers, analysts and leaders from around the world to San Francisco every year. Over four days, we shape the future of big data, analytics and AI as we share knowledge, hear from thought leaders and train on open-source technologies like Apache Spark, Delta Lake, MLflow, Koalas, TensorFlow and PyTorch. This year the virtual event for data teams has free general admission, 22-26 June.


Thursday, 2 April 2020

The Seattle Report on Data Research

I read with interest The Seattle Report on Data Research which presents the state of affairs for database systems. There are many interesting points raised about the diversity of the field and the challenges that are now apparent. These being:

  • Data Science having an emerging importance, 
  • HTAP (hybrid  transaction (OLTP) analytical processing (OLAP) creating real time operations to drive intelligent processing and reach, 
  • ethical responsibilities, 
  • scalable shareable data discovery 

This leads to the field having a bit of an identity crisis. We need to think about the end to end solutions that will provide insights whilst considering things such as data quality and data providence.

The question that we must all ask ourselves is, what is the vision for the future where data is all around us.


Tuesday, 17 March 2020

MVP Virtual Summit

The COVID-19 virus has resulted in Microsoft converting this year’s MVP/ Regional Director Summit to an online-only / virtual event. It was scheduled for the same week (Mar 16-19). Microsoft have worked hard to plan and prepare an alternate experience that still provides the exclusive in-depth technical sessions in an online-only format and is inclusive to our globally distributed community.

After day 2 of the summit it is still an amazing event. Using Microsoft Teams has been brilliant and the team have done an amazing job bringing it all together in such short notice.


Wednesday, 11 March 2020

Data Relay postpones to help SQLBits, a united community

It is a difficult time for conferences with the COVID-19 virus outbreak. As a conference owner we we have to think about the #SQLFamily and how to help keep everyone as safe as possible. It is with regret that we, Data Relay had to postpone our event in September, however I was proud to be part of a team that worked with a united community front. We have enabled SQLBits to move dates to 29th September - 3rd October, the original Data Relay dates. We now have to plan for a new set of Data Relay events and will share these once we have sorted this out.


.

Wednesday, 4 March 2020

International Women's Day 2020.

I am proud to be running an event for international women's day #IWD2020  this week. It is very important to have Diversity and Inclusion in the workplace. International Women's Day is on 8 March across the globe. Every year it is celebrated with a particular theme. The theme for this years international women's day is an equal world is an enabled world.


Thursday, 27 February 2020

SQL Bits 2020 Microsoft sessions

Here is a quick guide to a few of the Data and AI sessions from Microsoft at SQLBits.
Keynote: Join Gayle Sheppard, CVP, Azure Data for a keynote presentation on the journey of SQL Server from edge to cloud, packed with exciting demos and customer stories.
SQL Server 2019: The modern data platform: Bob Ward will give you the highlights, including the most exciting new features in SQL Server 2019 like Data virtualization, Big Data Clusters, Accelerated Database recovery, and more.
Sandboxing SQL Server 2019 Big Data Clusters: James Rowland Jones Big Data Clusters combine the power of SQL Server, Apache Spark, and HDFS to gain intelligence over all your data.
Modern Database design: Connor Cunningham will talk about improving the performance and scale of the databases in real-world scenarios using technologies including columnstore, in-memory tables, Azure SQL Database Hyperscale, Always On Availability Groups, and readable secondaries.
Azure Data Services on Azure Arc: Vin Yu talks about the newly launched hybrid service Azure Arc, which lets your run Azure Data Services anywhere. He will show you some of the management capabilities of Azure Arc such as elastic scale and automatic patching.
Azure Synapse Analytics:   Michael Rys brings together big data and data warehousing through new technology and a unified development experience.
Azure Data Studio features in 20 minutes: Vicky Harp will provide a quick refresher on the new features in Azure Data Studio.
10 DBA tasks for Azure SQL: Alain Dormehl and Rie Irish talk on the 10 most common DBA tasks for Azure SQL and how they differ from on prem DBA tasks.
Azure SQL Hyperscale Deep Dive: Alain Dormehl, Davide Mauri, and Denzil Ribeiro discuss how Hyperscale can enable you to innovate with nearly limitless storage in a future-proof, world class, fully managed database.
Microsoft Learn will showcase the complete offering of training and certifications available to the Data & AI professionals. 


Friday, 21 February 2020

Data Relay 2020

I am excited to share the 2020 Data Relay tour details. We will be returning to Reading this year following a break last year. The dates and venues are:

Leeds Monday 28 September
Nottingham Tuesday 29 September
Birmingham Wednesday 30 September
Reading Thursday 1 October
Bristol Friday 2 October


Monday, 10 February 2020

Free Azure certification Self Assessment Tool

Studying for Microsoft exams always begins with reviewing the exam syllabus. I then start mapping out my learning path, finding tools such as courses, ebooks, Microsoft learn training and online conference or Microsoft videos.

To help getting started there is a certification 'Self Assessment Tool'. This tool will help you prepare for your next certification exam, by ensuring you study for each of the objective domains and skills covered on the certification exam.


Wednesday, 5 February 2020

What is a Lakehouse?

A new management paradigm has emerged that combines data lakes and data warehouses. Lakehouses are similar to data warehouses with structures and data management features. This is backed with the low cost storage that is used for data lakes.

A lakehouse has some key attributes:

  • Transaction support
  • Schema enforcement and governance
  • BI support
  • Storage is decoupled from compute
  • Openness
  • Support for diverse data types ranging from unstructured to structured data
  • Support for diverse workloads
  • End-to-end streamingThere is a great article to read which covers this in more depth The Data Lakehouse – Dismantling the Hype

Thursday, 30 January 2020

Diversity Dashboard

I came across this interesting tool, a diversity dashboard. A tool for monitoring diversity and inclusion in your workplace using the workflow tools GitHub and Slack. The Diversity Dashboard is a beta tool we has been developed to offer insights into gender inclusion within the workplace.

There is  project underway to gain insight for diversity and inclusion. The Women in Data Science and AI research project at The Alan Turing Institute is working to map the participation of women in data science and AI in the UK, and to investigate diversity and inclusion in tech workplace cultures, and bias in the design of AI systems. The project aims to increase the number of women in data science and AI through the development of data-based policy recommendations, thereby addressing the risks posed by the lack of diversity in these fields.



Wednesday, 29 January 2020

SQLBits 2020 Theme

The SQLBits theme has been announced for 2020. The Greatest Data Show!

They say no one ever makes a difference being like everyone else. SQLBits want you to bring your alter ego, embrace oddity, and soak in the spectacle at #SQLBits2020!

Registration is still open, so register now.


Saturday, 25 January 2020

Carbon negative

I read with great interest about the Microsoft announcement of becoming carbon negative. Microsoft have stated they will be carbon negative by 2030. Also by 2050 Microsoft will remove from the environment all the carbon the company has emitted either directly or by electrical consumption since it was founded in 1975.

It is great that such an ambition has been set and it will be interesting to watch how this is achieved.  


Wednesday, 22 January 2020

Azure Backup support files restore for SQL Server 2019

Azure Backup protects standalone SQL Server and SQL AlwaysOn configurations in Azure Virtual Machine without the need to deploy and manage any backup infrastructure.  This is a key addition to the general availability of Azure Backup for SQL Server Virtual Machine . It is possible to restore .bak files. Read more about this restore operation.






















Saturday, 18 January 2020

Azure SQL Database Edge

Azure SQL Database Edge is available in public preview. Azure SQL Database Edge runs on ARM and Intel architecture and brings the Microsoft SQL engine to the edge. This advancement helps reduce the development required with the same Microsoft SQL database engine both on-premises and in the cloud. 


Wednesday, 15 January 2020

Microsoft Ignite The Tour 2020

Tomorrow it is Microsoft Ignite The Tour 16-17 January 2020 in London. Microsoft Ignite The Tour provides technical training led by Microsoft experts and your community. There are plenty of exciting things to learn, new ways to build solutions, migrate and manage infrastructure, and connect with local industry leaders and peers.


Tuesday, 7 January 2020

ODI Data Ethics Canvas

The ODI have created a useful tool for anyone who collects, shares or uses data. This aims to help identify management ethical issues at the start of a data project. The Open Data Institute defines data ethics as:

'A branch of ethics that evaluates data practices with the potential to adversely impact on people and society – in data collection, sharing and use'

They say data ethics relates to good practice around how data is collected, used and shared. It is especially relevant when data activities have the potential to impact people and society, directly or indirectly.

The Data Ethics Canvas is a part of a wider data toolkit.


Thursday, 2 January 2020

Asilomar AI Principles

Data Ethics has been brought to the fore by AI algorithms showing bias. There are various insightful articles which discuss data ethics. The Asilomar Conference on Beneficial AI organized by the Future of Life Institute was held January 5-8 2017 at the Asilomar Conference Grounds in California. The conference aimed to address and formulate principles of beneficial AI. With more than 100 thought leaders and researches in economics, law, ethics and philosophy at the conference, it resulted in the creation of a set of guidelines for AI research. There are 23 Asilomar AI Principles of which many are related to ethics and values.





























This is a significant enhancement on the Isaac Asimov's "Three Laws of Robotics" which were shared in his 1942 short story "Runaround". The Three Laws he listed were:

  • A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  • A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
  • A robot must protect its own existence as long as such protection does not conflict with the First or Second Laws.

In 2016 Satya Nadella did share a vision for more relevant AI rules.

  • AI must be designed to assist humanity.
  • AI must be transparent. 
  • AI must maximize efficiencies without destroying the dignity of people. 
  • AI must be designed for intelligent privacy. 
  • AI must have algorithmic accountability. 
  • AI must guard against bias. 

From this it has led to data ethics becoming its own branch of ethics.