Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein

Tuesday, 28 April 2015

SQL Saturday Exeter 2015

SQL Saturday Exeter was held Apr24-25 2015 at Jurys Inn Hotel in Exeter. I was excited to be helping at the event for the community again.  There were various excellent training day sessions. 

The Saturday keynote was the most amazing fun keynote I have seen and a highlight of the event. The video was by Phil Factor about Spinach and Database Development. 

Here a few of the points contained within the video on Spinach:

What has spinach got to do with Database Development? Generations of children were fed spinach in preference to more nutritious things, such as cardboard, because of the persistence of bad data.

Guarding the quality of your data is about the most important thing that you as a data professional can do. You may think that performance is important but it would just deliver you the wrong answer faster. Resilience? it would just make it more likely that you’d be able to deliver the wrong answer. Delivery? Yep you got it, the wrong answer quicker.

A good database professional must be unequivocal.

Statistics can help us to make very confident assertions about large datasets if they conform to one of the common distributions but they cannot tell us anything about individual items of data. You can of course remove outliers but in fact outliers are just items of data that don’t conform to your assumptions about the data and the whole point of data analysis is to test your assumptions. By cleaning data, by removing outliers you can prove almost anything scientifically.

I would recommend watching the keynote.


Saturday, 14 March 2015

Free Power BI Designer Desktop App

The new PowerBI dashboard designer in early preview in the US provides data transformation and visual analytics to build reports. It can be downloaded from powerbi.com 

Power BI Designer consolidates Power BI Excel add-in tools: Power Query, Power Pivot and Power View into one application.

Power BI Designer also provides self-service ETL . The free desktop data discovery tool can be downloaded and installed side-by-side with any version of Office or Excel on Windows.

More details are in the great article by Jen Underwood at  http://sqlmag.com/power-bi/introducing-free-power-bi-designer-desktop-app

The use of power BI tools is discussed by James Serra  with tools defined as

•Front-end (Excel) 
•Data shaping and cleanup. Self-service ETL (Power Query) 
•Data analysis (Power Pivot) 
•Visualization and data discovery (Power View, Power Map, Power BI Designer)
• Dashboarding (Power BI Dashboard)
• Publishing and sharing (Power BI sites)
• Natural language query (Power BI Q&A)
• Mobile (Power BI for Mobile)
• Access on-premise data (DMG, Analysis Services Connector) Power Query Power Pivot Power View Power Map Power BI Designer Power BI Dashboard Power BI Site Power BI Q&A Power BI for mobile

James displays the data flow as


Sunday, 8 March 2015

SQLBits Superheroes

SQLBits Superheroes was held at ExCeL, Royal Victoria Dock, London between 4-7 March 2015. In its usual tradition an amazing conference packed full of a massive variety of Data and Database technology sessions. Attendees came from all over the world and it was a privilege to be able to help at the event again. I have now attended every SQLBits event and seen the event grow in size and strength. It is the place for the SQL community to grow and share.  

This year SQLBits was extended by an extra day consisting of 3 days and a free SQL Server community day on Saturday. There were more preconference choices, extended session lengths and double length sessions. The variety of topics discussed has exploded to cover the vast breath of the data platform from OLTP, Data Warehousing, Analytics, Hadoop, Azure etc. for DBA’s, developers and analysts.  All in all an amazing conference.

Wednesday, 4 February 2015

Data Visualization Tools

A couple of Data Visualization tools to help with analysizing structured and unstructured .


This is Elasticsearch’s data visualization engine. It allows native interaction with all  data in Elasticsearch via custom dashboards. It has dynamic dashboard panels displaying changes to queries in real-time. Kibana works with any kind of data — structured and unstructured. It helps understand large volumes of data, easily create bar charts, line and scatter plots, histograms, pie charts, and maps.


Banana is the open source port of Kibana 3 and is available on GitHub at https://github.com/LucidWorks/banana/

Banana is a tool that allows you to create dashboards to visualize data stored in Solr.

The Banana dashboard has several panels that show the data

Saturday, 24 January 2015

Management of Database Systems Research, a précis on Technet

 Graphics by @DeepFat.

I am very pleased to announce that an article introducing my research on A Study into Best Practices and Procedures used in the Management of Database Systems entitled Management of Database Systems Research, a précis is published on the TechNet UK Blog.  http://bit.ly/1yMvzzS

There is a fun cartoon accompanying the article by an amazing illustrator, Andrew Fryer.

Thank you Charlotte Utting @TechNetUK

Saturday, 10 January 2015

Database Research Publication

I have been working on my part time research, at The Open University, for my doctorate for a number of years which is a study into best practices and procedures used in the management of database systems
For the start of any research it is necessary to accurately baseline the field of study. This baseline was created through a worldwide quantitative survey to examine to what extent  best practices and procedures were utilised by the database community for the management of database systems. 453 respondents kindly contributed to the survey. Part of this research from the quantitative stage is now published in the Information Systems journal. 

An Audio Slides presentation is available introducing the paper http://www.sqltoolkit.co.uk/publications.html  

The formal reference:
Holt, V. et al., 2015. The usage of best practices and procedures in the database community. Information Systems, 49, pp.163–181. Available at: http://linkinghub.elsevier.com/retrieve/pii/S0306437914001914
DOI: http://dx.doi.org/10.1016/j.is.2014.12.004

The free link is valid until February 24, 2015 http://bit.ly/1zqTjLo

More information about the research can be found at http://www.sqltoolkit.co.uk/ 

Friday, 2 January 2015

Data and Database Predictions for 2015

There are many predictions of what data and database advancements will transpire throughout 2015. These are my favourite data and database predictions for 2015.

Top 10 2015 Predictions for Microsoft Azure

PAAS will provide a more economical… the case for IAAS based services is being undermined by easier to use PAAS services.

Why 2015 Will Be Year of Big Data: Oracle's Seven Predictions

Data is a new form of capital. Ultimately, information about people, places and things will truly differentiate enterprises.

IDC Reveals Worldwide Big Data and Analytics Predictions for 2015

Growth in applications incorporating advanced and predictive analytics, including machine learning, will accelerate in 2015. These apps will grow 65% faster than apps without predictive functionality.

Database Predictions for 2015

SQL and NoSQL will become best friends

10 Predictions for Data and Analytics in 2015

Hybrid architectures will become the norm for many organizations,

Gartner Identifies the Top 10 Strategic Technology Trends for 2015

Analytics will take center stage as the volume of data generated by embedded systems increases and vast pools of structured and unstructured data inside and outside the enterprise are analyzed.

Your top predictions for Business Analytics in 2015

LinkedIn Data Culture Discussions: Data storytelling will become the next 'big thing' in Analytics!