Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein

Monday, 10 July 2017

Inspire Opening Keynote

I watched the Inspire keynote yesterday. It started with  @MelissaArnot (the first American woman to summit Everest without supplemental oxygen) talking about following her dreams with determination and collaboration. Dreams are a global language. Her introductory speech was so inspiring, encouraging you to never give up and to press on amidst adversity, to reach your goal.

Microsoft talked about the Intelligent cloud and intelligent edge. Technological changes are no longer bound to a single device. There is a profound shift to include AI in every application that is built. Data has gravity. It is about managing complexity on a new efficient frontier.

The new changes will be more profound that virtualisation. The new pillars for enabling digital transformation are: the modern workplace, business applications, applications & infrastructure and data & AI. It is about democratisation and making use of data and service, which are now digital. It is important to incorporate the customers needs in the solutions that we create.

Microsoft introduced a new intelligent solution, Microsoft 365. The modern workplace is shaping culture and it is necessary to empower people, to help share and transform, in disparate teams. It needs to be one simple framework or integrated system.

There were various other announcements but it is Data & AI changing model, that excites me.  Serverless and Microservices will be a revolutionary change but it is the fact that AI will be built into all services, that will define the next era.   Incorporating database richness, connecting into one digital data estate is the future, although it is not just about big data and analytics. It is about now trying to create value with small data.

Without data there is no AI. Data is the foundation.

It is not possible to succeed alone, so choosing your Partners is crucial. 

Saturday, 8 July 2017

The Conference Year

Conferences add significant business value: from gaining knowledge, to problem solving, to learning about what the future has to offer and networking. Networking is a key enabler.  The question is always which conferences to attend and which keynotes to watch. The ones you attend can drive your future career path. So it is an important choice, particularly when there is so much choice. The choice could be about technologies and industries that you think will define the next decade and beyond or to help you with your present day tasks. I like to attend conferences for both reasons and the community. They enable you to "image" and "innovate".

I help at two community run conferences. These two conferences continue to inspire me, enable me to be better and to be a part of the community.

Thursday, 6 July 2017

Microsoft Faculty Summit 2017: The Edge of AI

Research is at the forefront of innovation and drives the future. Blue-sky thinking is critical for new ideas to become reality. The 18th Annual Microsoft Research Faculty Summit is on July 17-18, 2017 and looks to have an interesting agenda. The theme is "The Edge of AI". It will reflect on the current state of AI and will discuss the research, the tools, the services, and the community engagement that the research community need to invest in to help accelerate innovation and democratize AI, in order to solve the world’s most pressing challenges. Microsoft will highlight some of the key challenges posed by artificial intelligence, and will identify the next generation of approaches, techniques, and tools that will be needed to develop AI to solve the world’s most pressing challenges.

It will be streamed live 8:30 AM Pacific Time (UTC-7)

Tuesday, 4 July 2017

Database Now! Online Webinars

Database Now Online is on 19th July 2017 between 8:00 am and 2:30 pm (Pacific Time). This free online event is a full day of live webinar sessions presented by industry experts, each designed to teach Architects, Modellers, Database Administrators, and IT Managers about the latest in enterprise solutions for data storage, data access, scaling out, and cloud based solutions. There looks like a raft of very interesting sessions to watch.

You can register now

Saturday, 1 July 2017

Journey to the cloud

I have been reading a very interesting free ebook Enterprise Cloud Strategy by Barry Briggs and Eduardo Kassner. 

There are various types of modernisation discussed. 

It is based on Based on “Gartner Identifies Five Ways to Migrate Applications to the Cloud”, Gartner Inc., 2011. 

The overview of cloud migration principles discussed are summarised:

There is new a model that truly adopts cloud taking into account SaaS, PaaS and IaaS. This new architecture principle means considering the best placement of workloads, always looking at SaaS first.

This new way of thinking follows this placement decision tree which helps you make the correct decision on whether to use SaaS, PaaS, IaaS or private cloud.

On the data side, the paper talks about dividing your data into several categories related to risk. This is really about creating your own business data DNA. 'Data analytics' and 'BI and analytics'  cloud architectural blueprints are shared.

An interesting change for roles is the evolution of roles for the cloud. Microsoft suggest the data roles of Data Scientist and Information Architect emerge from the DBA and Statistician.

Thursday, 29 June 2017

AWS Summit 2017 London

Yesterday was my first AWS Summit at ICC London, ExCeL. The event was much larger than I thought. On arrival I had to queue for over 25 minutes to get into ICC London. Then had to queue again to collect the registration badge, then had to go to another room to be security checked. 45 minutes later managed to get a seat in the keynote which had already started.

There was a great demo showing robots working in a distribution centre at @OcadoTechnology using AI. The keynote raised the question of old world relational databases. Do we still need old world relational databases? They say not and MySQL, PostgreSQL and MariaDB are why customer are moving to Open database engines.

The Forces and market trends that are causing adoption of new services

Amazon has various database offerings.
  • Amazon Aurora (a MySQL-compatible relation database engine that combines the speed and availability of high-end commercial databases)
  • Amazon Relational Database Service (covers MySQL, Oracle, SQL Server, or PostgreSQL database in the cloud)
  • Amazon DynamoDB (a highly scalable, fully managed NoSQL database service)
  • Amazon Redshift (a fully managed, easily scalable petabyte-scale data warehouse service that works with your existing business intelligence tools)
  • Amazon ElastiCache (a web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud.)
  • AWS Database Migration Service

Migrating Data to and from AWS

When migrating databases to AWS you can backup your SQLServer database and restore the backup file to AWS RDS or EC2. The migration steps

There were various things to learn from the migration.

The Amazon Relational Database Service (Amazon RDS) covers

Amazon Aurora keeps 6 copies of the data, two copies in each availability zone to protect against availability zone failure.

There are various Analytics services such as
  • Amazon Athena (is an interactive query service to analyze data in Amazon S3 using standard SQL. Athena is serverless)
  • AWS Data Pipeline (is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premise data sources, at specified intervals)
  • AWS Glue (is a fully managed ETL service that makes it easy to move data between your data stores)
  • Amazon Kinesis (to collect, process, and analyse real-time streaming data) 

Artificial Intelligence service offerings are: 
  • Amazon Lex
  • Amazon Polly
  • Amazon Rekognition
  • Amazon Machine Learning
  • AWS Deep Learning AMIs
  • Apache MXNet on AWS

AWS was called the centre of gravity for AI.

Machine Learning capabilities explained

Friday, 2 June 2017

Azure Application Architecture Guide

I came across this useful guide. It presents a structured approach for designing applications on Azure that can scale, are resilient, and highly available. It is based on proven practices that Microsoft have learned from customer engagements.

Thursday, 1 June 2017

Continual Growth of DTU

Database Transaction Unit (DTU) 

Database Transaction Unit (DTU) is a blended measure of CPU, memory, I/O (data and transaction log I/O). For a single Azure SQL database at a specific performance level within a service tier, Microsoft guarantees a certain level of resources.  This was originally calculated by OLTP benchmarks.

Azure SQL Database Advisor enables deeper insight into the DTU resource consumption.

A useful tool to help calculate the DTU size required for the service in Azure SQL Database is the DTU Calculator.

Measure Resource Utilization

To measure resource utilization for your existing database server, you'll need to capture several windows performance metrics on your SQL Server. It is best to run the performance counters for a while to obtain a representative workload pattern. This should be no less than an hour to provide the best recommendation. The counters to use are:

  • Processor - % Processor Time
  • Logical Disk - Disk Reads/sec
  • Logical Disk - Disk Writes/sec
  • Database - Log Bytes Flushed/sec

Elastic Database Transaction Units (eDTUs)

There is another type of DTU and that is elastic DTU. Rather than providing a dedicated set of resources (DTUs) to a SQL Database you can place the databases into an elastic pool. This is where a SQL Database server shares a pool of resources among databases. Elastic pools provide a simple cost effective solution to manage the performance for multiple databases that have widely varying and unpredictable usage patterns.

A pool is given a set number of eDTUs for a set price.

Change in Elastic Pool Sizes

There has been a change now with the availability of higher database eDTU limits for standard elastic pools in Azure SQL Database. The standard elastic pool was limited to 100 eDTUs but, it is now increased up to 3000 eDTUs.

Friday, 26 May 2017

Azure Red Shirt Dev Tour '17

I attended the Azure Red Shirt Dev Tour on 25 May at the Live Room, Twickenham Stadium presented by Scott Gutherie. The day was packed with an end to end view of Azure showing how the technology can be used, to business advantage, to build the next generation applications. They also show cased the announcements from the Microsoft Build Conference, the week before. The after lunch session went through Azure tips and tricks to help get the most out of the platform.

It was an amazing day full of new technology. My favourite new service has to be Azure Cosmos Database, 4 data models in 1.

Data Amp Reading

I will be attending the UK launch of SQL Server 2017 on 31 May 2017 at Reading. The agenda looks interesting covering details of how to innovate and transform your business. The Microsoft’s Data Platform topics include:

  • Azure Data Services 
  • Business Analytics
  • SQL Server including Linux support in SQL Server v.Next

The aims of the event are to help enable :

  • optimisation of a data strategy from on-premises to cloud
  • modernise to the latest mission critical on-premises and cloud solutions
  • build intelligent applications using Microsoft Cognitive Services
  • transform the landscape of data technology, so that you can make the right decisions
  • learn how to transform using advanced analytics, machine learning and artificial intelligence
  • amplify the impact by building apps that matter with unmatched performance and security 

Sunday, 21 May 2017

Azure SQLDB Managed Instance and Database Migration Service

There are 2 services in preview at the moment Azure SQLDB Managed Instance and Database Migration Service. Both these services seem very useful for moving to the cloud.

They enable migration of a fleet of SQL Server databases without application re-architecture to the cloud.

Azure SQLDB Managed Instance provides security isolation with Azure Virtual Network, along with built-in HADR, built-in intelligent performance tuning, and intelligent security services that are already available in Azure SQL Database. No public endpoint is required.

Databases up to 35TB can be migrated.

It can be set up with a simple restore of native databases or log shipping using Azure Database Migration Service

Azure SQLDB Managed Instance provides SQL Server application compatibility including items such as commonly used cross database references using three-part names, CLR and R Services, DB Mail, SQL Agent, Transactional Replication, Change Data Capture and Service Broker.

Thursday, 11 May 2017

Azure managed database services extended

There were several developer announcements at the Microsoft Build conference for Azure data and cloud services to speed up modernization. Microsoft has now extended Azure managed database services with the introduction of MySQL and PostgreSQL.

This gives developers expanded choice and flexibility. 

More details of the announcements can be read below.

Top 10 reasons Day 1 of Build excited developers and Cortana, too