Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Friday, 2 June 2017

Azure Application Architecture Guide

I came across this useful guide. It presents a structured approach for designing applications on Azure that can scale, are resilient, and highly available. It is based on proven practices that Microsoft have learned from customer engagements.


Thursday, 1 June 2017

Continual Growth of DTU

Database Transaction Unit (DTU) 

Database Transaction Unit (DTU) is a blended measure of CPU, memory, I/O (data and transaction log I/O). For a single Azure SQL database at a specific performance level within a service tier, Microsoft guarantees a certain level of resources.  This was originally calculated by OLTP benchmarks.













Azure SQL Database Advisor enables deeper insight into the DTU resource consumption.

A useful tool to help calculate the DTU size required for the service in Azure SQL Database is the DTU Calculator.

Measure Resource Utilization

To measure resource utilization for your existing database server, you'll need to capture several windows performance metrics on your SQL Server. It is best to run the performance counters for a while to obtain a representative workload pattern. This should be no less than an hour to provide the best recommendation. The counters to use are:

  • Processor - % Processor Time
  • Logical Disk - Disk Reads/sec
  • Logical Disk - Disk Writes/sec
  • Database - Log Bytes Flushed/sec

Elastic Database Transaction Units (eDTUs)

There is another type of DTU and that is elastic DTU. Rather than providing a dedicated set of resources (DTUs) to a SQL Database you can place the databases into an elastic pool. This is where a SQL Database server shares a pool of resources among databases. Elastic pools provide a simple cost effective solution to manage the performance for multiple databases that have widely varying and unpredictable usage patterns.




A pool is given a set number of eDTUs for a set price.

Change in Elastic Pool Sizes

There has been a change now with the availability of higher database eDTU limits for standard elastic pools in Azure SQL Database. The standard elastic pool was limited to 100 eDTUs but, it is now increased up to 3000 eDTUs.

Friday, 26 May 2017

Azure Red Shirt Dev Tour '17











I attended the Azure Red Shirt Dev Tour on 25 May at the Live Room, Twickenham Stadium presented by Scott Gutherie. The day was packed with an end to end view of Azure showing how the technology can be used, to business advantage, to build the next generation applications. They also show cased the announcements from the Microsoft Build Conference, the week before. The after lunch session went through Azure tips and tricks to help get the most out of the platform.

It was an amazing day full of new technology. My favourite new service has to be Azure Cosmos Database, 4 data models in 1.





Data Amp Reading














I will be attending the UK launch of SQL Server 2017 on 31 May 2017 at Reading. The agenda looks interesting covering details of how to innovate and transform your business. The Microsoft’s Data Platform topics include:

  • Azure Data Services 
  • Business Analytics
  • SQL Server including Linux support in SQL Server v.Next

The aims of the event are to help enable :

  • optimisation of a data strategy from on-premises to cloud
  • modernise to the latest mission critical on-premises and cloud solutions
  • build intelligent applications using Microsoft Cognitive Services
  • transform the landscape of data technology, so that you can make the right decisions
  • learn how to transform using advanced analytics, machine learning and artificial intelligence
  • amplify the impact by building apps that matter with unmatched performance and security 

Sunday, 21 May 2017

Azure SQLDB Managed Instance and Database Migration Service


There are 2 services in preview at the moment Azure SQLDB Managed Instance and Database Migration Service. Both these services seem very useful for moving to the cloud.

They enable migration of a fleet of SQL Server databases without application re-architecture to the cloud.

Azure SQLDB Managed Instance provides security isolation with Azure Virtual Network, along with built-in HADR, built-in intelligent performance tuning, and intelligent security services that are already available in Azure SQL Database. No public endpoint is required.

Databases up to 35TB can be migrated.

It can be set up with a simple restore of native databases or log shipping using Azure Database Migration Service

Azure SQLDB Managed Instance provides SQL Server application compatibility including items such as commonly used cross database references using three-part names, CLR and R Services, DB Mail, SQL Agent, Transactional Replication, Change Data Capture and Service Broker.

Thursday, 11 May 2017

Azure managed database services extended

There were several developer announcements at the Microsoft Build conference for Azure data and cloud services to speed up modernization. Microsoft has now extended Azure managed database services with the introduction of MySQL and PostgreSQL.



This gives developers expanded choice and flexibility. 

More details of the announcements can be read below.

Top 10 reasons Day 1 of Build excited developers and Cortana, too 

Wednesday, 10 May 2017

Azure Cosmos DB

Another database annoucment today from Microsoft. The first globally distributed, multi-model database service. Microsoft describe Azure Cosmos DB as containing a write optimized, resource governed, schema-agnostic database engine that natively supports multiple data models: key-value, documents, graphs, and columnar.





















Azure Cosmos DB is an evolutionary leap for DocumentDB which states it contains APIs for accessing data including MongoDB, DocumentDB SQL, Gremlin (preview), and Azure Tables (preview).



Azure Cosmos DB contains a write optimized, resource governed, schema-agnostic database engine that natively supports multiple data models: key-value, documents, graphs, and columnar. 

The Key Capabilities
  • Turnkey global distribution
  • Multiple data models and popular APIs for accessing and querying data
  • Elastically scale throughput and storage on demand, worldwide
  • Build highly responsive and mission-critical applications
  • Ensure "always on" availability
  • Write globally distributed applications, the right way
  • Money back guarantees
  • No database schema/index management
  • Low cost of ownership

Tuesday, 2 May 2017

Insights on SQLBits











If I could pick 4 words that sum up SQLBits it would be: 

Data. Inspire. Community. European

The Conference

SQLBits is the largest SQL Server event in Europe. It is run by volunteers in the community and SQL Server enthusiasts.  The event has grown over the years and is now a 4 day event.
  • Wednesday pre-con training day
  • Thursday pre-con training day followed by a pub quiz
  • Friday is deep dive session day concluding with the conference fancy dress party
  • Saturday is the free community day event.
The Days
Every day is packed full of Microsoft SQL Server and Microsoft Azure content in all forms. SQLBits now covers content from database management, development, data management, business intelligence and data science. In true tradition, the community gets the chance to vote for the huge variety of sessions.  There is also the chance to attend some sessions on professional development as well.

The Announcement
Every year there is excitement to find out the dates of the event, the location of the event and the theme of the event.

The People
There is the opportunity to meet some of the brilliant minds in the field and to be dazzled by the continual suite of innovative changes. There are a variety of speakers from Most Valued Professionals (MVPs) to new speakers from the SQL community. There are also the conference helpers who enjoy helping make the conference a success. 

The Community
There is always the hustle and bustle of the attendees in the sponsor hall. The sponsors of the event provide crucial funding for the conference and showcase their latest products and services. Then there are the attendees who come to learn, be motivated and network.

The Theme
Every event has a theme. Some of the themes have ranged from James Bond, Robin Hood, Steampunk, Superheroes, Space to Disco. 

The Event
This year, the event returned to Telford near the birthplace of the industrial revolution. This is quite fitting with 2017 being the year of the algorithm, providing the intelligence revolution.

Monday, 1 May 2017

Interesting blog posts of the last month

These are a few blog posts I read last month which i found quiet interesting.

Azure SQL 
Resource Manager template reference now available
DEA 2.0 Technical preview: Release Overview – Database Experimentation Assistant: Overview
Azure Data Warehouse: Understanding the DWU, workload management, resource and memory allocation
Top 30 SQL Blogs  Websites for Microsoft SQL Server Administrators
Microsoft touts SQL Server 2017 as the first RDBMS with built-in AI

NoSQL
AWS Database Migration Service expands into NoSQL migrations with MongoDB & Amazon DynamoDB support

DocumentDB

Visualization

Getting started with R in SQL Server

Deep Learning

Graph
Build a recommendation system with the support for graph data in SQL Server 2017 and Azure SQL DB

Data

Book
The Data and Analytics Playbook: Proven Methods for Governed Data and Analytic Quality 1st Edition