GDPR

The General Data Protection Regulation (GDPR) comes into effect on 25 May 2018, one month from now. The EU General Data Protection Regulation is the most important change in data privacy regulation in 20 years. GDPR, is fundamentally about protecting and enabling the privacy rights of the individual.

A Guide to enhancing privacy and addressing GDPR requirements with the Microsoft SQL platform is an interesting read. The obligations related to controls and security around handling of personal data are some of the the concepts discussed in the the document.

GDPR Article 25—“Data protection by design and default”: Control exposure to personal
data.
• Control accessibility—who is accessing data and how.
• Minimize data being processed in terms of amount of data collected, extent of
processing, storage period, and accessibility.
• Include safeguards for control management integrated into processing.
GDPR Article 32—“Security of processing”: Security mechanisms to protect personal data.
• Employ pseudonymization and encryption.
• Restore availability and access in the event of an incident.
• Provide a process for regularly testing and assessing effectiveness of security
measures.
GDPR Article 33—“Notification of a personal data breach to the supervisory authority”:
Detect and notify of breach in a timely manner (72 hours).
• Detect breaches.
• Assess impact on and identification of personal data records concerned.
• Describe measures to address breach.
GDPR Article 30—“Records of processing activities”: Log and monitor operations.
• Maintain an audit record of processing activities on personal data.
• Monitor access to processing systems.
GDPR Article 35—“Data protection impact assessment”: Document risks and security
measures.
• Describe processing operations, including their necessity and proportionality.
• Assess risks associated with processing.
• Apply measures to address risks and protect personal data, and demonstrate
compliance with the GDPR.

SQL Information Protection with Data Discovery and Classification

The public preview of SQL Information Protection brings advanced capabilities built into Azure SQL Database for discovering, classifying, labeling, and protecting the sensitive data in your databases. SQL Data Discovery and Classification are also added to SQL Server Management Studio.

This tools will help meet data privacy standards and regulatory compliance requirements, such as GDPR. It will enable data-centric security scenarios, such as monitoring (auditing) and alerting on anomalous access to sensitive data to be viewed in dashboards. It will help with controlling access to and hardening the security of databases containing highly sensitive data.

The SQL Information Protection (SQL IP) introduces a set of advanced services and new SQL capabilities, forming a new information protection paradigm in SQL aimed at protecting the data. The four areas covered are:

• Discovery and recommendations 
• Labeling
• Monitoring/Auditing (Azure SQL Db only)
• Visibility

Demystify complex relationships with SQL Server 2017 and graph

This great infographic shows some quick tips about SQL Server 2017 and graph databases. You can view this at: http://msft.social/jCIE18 . The picture demonstrates nodes and edges and provides a clear example of the code changes between the Traditional SQL query and the Graph query.

SQL Operations Studio Cheat Sheet

Andy Mallon has created a new version of his very useful cheat sheet of shortcuts for SQL Operations Studio. You can download the new version here.

Microsoft SQL Server Machine Learning

I am pleased to be mentioned in a TechTarget article.

PASS Summit 2017 Day 1 Keynote

I attended PASS Summit 2017 which was my second year of attendance. I enjoyed the conference enormously. It is enjoyable being immersed in data and being with people who are enthusiastic in the field.

The Day 1 Keynote "Microsoft for the Modern Data Estate" was presented by Rohan Kumar. Data is driving transformation. Data, Cloud and AI are the three most disruptive trends of our time. The modern data estate, enables simplicity and common sense. It takes any data from any source, structured or unstructured data and large or small data. The modern data estate provides a seamless infrastructure between on premises, private and public cloud, enabling a hybrid set up that hides the dichotomy of these disparate systems. Seamless flexibly and a choice of engines.

New features in SQL Server 2017

There are many changes to SQL Server 2017. SQL Server 2017 has industry leading performance and security now on Linux and Docker. The key engine changes 

• Support for graph data and queries
• Advanced Machine Learning with R and Python
• Native T-SQL scoring
• Adaptive Query Processing and Automation Plan Correction

SQL Server 2017 will enable deployment in seconds on Linux and Windows containers and has special pricing for SQL Server on Linux and Red Hat Enterprise Linux.

New Features Azure SQL Database

Azure SQL Database offers intelligent DBaaS, privacy and trust, seamless and compatibility and competitive TCO. There is seamless migration to the cloud with the cloud first approach breading faster innovations. The list of changes presented

Azure Data Factory now provides a managed environment for SQL Server Integration Services (SSIS) packages and easily move your SSIS workloads to cloud.

There was the announcement made for a new tool called, Microsoft SQL Operations Studio, a free lightweight modern data operations tools for SQL everywhere.

These are but some of the changes coming to the products.

Microsoft for the Modern Data Estate

The Microsoft Ignite session on the modern data estate was full of announcements. All around us, data is driving digital transformation. Modernize with SQL Server 2017 on Linux and Windows and Azure Data Services; deliver modern intelligent applications using technologies like Azure Cosmos DB and Azure Database for PostgreSQL.

The world is changing.  We need to help invest in the future without being tied to the past. AI is a fundamental pillar to leverage that. If businesses invests in data they outperform other companies. 

Data doesn’t need to leave the database for data science to take place.

The cloud first approach breeds faster innovation and SQL Server 2017 is proof of that.

Announcements

SQL Server 2017 on Linux, Docker , and Windows server

Supports for graph data and queries

Advances Machine Learning with R & Python

Native T-SQL scoring

Adaptive Query processing and Automatic Plan Correction

Vulnerability assessment for GDPR - preview

Intelligent insights into performance – preview

Support for Graph data and queries –GA

Adaptive query processing – GA

Native scoring and support for Azure Machine Learning - GA

SQL Database and Database Migration Service

Migration to the cloud is easy with this new service. 

Azure SQL Database is the intelligent cloud database for app developers. It learns and adapts, scales on the fly, enables multi-tenant SaaS apps, works in your environment, secures and protects. The systems of intelligence on SQL Database is shared.

Globally Distributed Applications

Announcing Azure functions for Azure Cosmos DB to build apps faster with a serverless infrastructure. 

Uncovering insights with big data and advanced analytics

The new Azure Data Factory allows easy modelling of diverse data integration scenarios. You can now with the preview service, easily move your SQL Server Integration Services (SSIS) workloads to cloud. There is also a data movements as a service with 30+ connectors. Azure SQL Data Warehouse has a compute- optimized tier and unlimited columnar storage in preview. The last announcement was the Power BI Report Server.

SQL Server 2017

Microsoft have announced the general availability of SQL Server 2017 and Machine Learning Services. SQL Server 2017 is available for purchase on October 2nd.

SQL Server 2017 is a milestone representing the first version of SQL Server to run on Windows Server, Linux and Docker. It has already has been pulled on Docker X 2 million times since November.

Data Amp Reading

I will be attending the UK launch of SQL Server 2017 on 31 May 2017 at Reading. The agenda looks interesting covering details of how to innovate and transform your business. The Microsoft’s Data Platform topics include:

• Azure Data Services 
• Business Analytics
• SQL Server including Linux support in SQL Server v.Next

The aims of the event are to help enable :

• optimisation of a data strategy from on-premises to cloud
• modernise to the latest mission critical on-premises and cloud solutions
• build intelligent applications using Microsoft Cognitive Services
• transform the landscape of data technology, so that you can make the right decisions
• learn how to transform using advanced analytics, machine learning and artificial intelligence
• amplify the impact by building apps that matter with unmatched performance and security 

Microsoft DataAmp – SQL Server 2017

The DataAmp webcast was packed full of announcements. The Webcast was delivered by Scott Guthrie and Joseph Sirosh. The SQL Server product delivering intelligence, trust and flexibility.

Microsoft confirmed that the next version of SQL Server is SQL Server 2017 and will be available simultaneously on Windows, Linux and Docker. Download the SQL Server 2017 datasheet.  It will be the first RDBMS to deliver AI with data. There is a convergence of cloud, data and intelligence. Delivering AI with data: the next generation of Microsoft’s data platform blog shares more information.  SQL Server 2017 Community Technology Preview 2.0 now available.

There were so many new features announced only a few are mentioned below.  There are adaptive query processing improvements which will enhance the performance of workloads. There is a You Tube video SQL Server 2017: Adaptive Query Processing discussing this. Threat detection is now in Azure SQL Database and is straight forward to configure. 

Hybrid Cloud just got easier to adopt with the new Azure migration resources and tools. To help with SQL Server migrations to the cloud features such as Service Broker, SQLAgent, Profiler etc. are now available in Azure. There is a new data migration service for automatic migration for SQL Server, Oracle and MySQL in Azure .

SQL Graph

Storing and analyzing graph data relationships. This includes full CRUD support to create nodes and edges and T-SQL query language extensions to provide multi-hop navigation using join-free pattern matching.  The SQL Server engine integration enables querying across SQL tables and graph data.

SQL Server on Linux

 

The official Microsoft repository for SQL Server in Docker containers is here.

Here are a few videos to help get you started with Linux:

• SQL Server 2017: Security on Linux
• Get started developing apps with SQL Server 2017 running on Linux
• SQL Server 2017: HA and DR on Linux

Analytics 

SQL Server is the first commercial database to include Deep Learning algorithms, with the announcement of the Microsoft Cognitive Services general availability of the FACE API and Computer Vision API.

Azure Data Lake Services now have petabyte scale.

Azure Analysis Services became generally available. You Tube video: SQL Server 2017: BI enhancements.  

You can use Python for advanced analytics, You Tube video: SQL Server 2017: Advanced Analytics with Python

Azure DocumentDB

Azure DocumentDB is globally distrubuted and offer limitless scale of throughput and storage. It can be used for things such as IoT applications that need low response times and need to handle massive amounts of reads and writes.

Cortana Intelligence Solution Templates

You can now quickly build Cortana Intelligence Solutions from preconfigured solutions, reference architectures and design patterns. Some are released with more to follow.

The really important thing that I am excited about is the flexibility of choice within the SQL Server product.

Joseph Sirosh concluded comparing the industrial revolution with the intelligence revolution of today.