Database Management Survey

The Database Management Survey is open and available to complete at  http://svy.mk/UcaxDm .

I am currently undertaking a PhD with the Open University and my research investigates the current practices and procedures that are used in database management and examines the complexities of managing database systems.  The Database Management Survey is for database administrators, data professionals etc to complete for every database engine.

It is a comprehensive survey that covers the end to end lifecycle for managing database systems across on premises, cloud, differing database engines, virtualization and big data etc. The survey will take about 15 minutes to complete and I would be most grateful if you could complete as many questions as possible.

Many thanks for your help with the Database Management Survey 2013 and Merry Christmas.

@victoria_holt

Management of Database Systems Survey

I am a senior certified Microsoft Certified IT Professional (MCITP) Database Administrator who is examining how management of database systems can be improved for the database community, as part of my PhD study with The Open University. To proceed I need your help and the help of as many people as possible in the industry. We see a lot of discussion about new database engines, tools, features, the cloud, big data etc. but how do we administer and manage this ubiquitous landscape?

I need to know to what extent practices and procedures are utilised by the database community and what affects their adoption. Please help me with my research by completing this survey. It will take 15-20 minutes of your time and it is an in depth survey. There are no prizes for helping, just the satisfaction that you have helped the database community as a whole and of course my gratitude. Thank you very much for agreeing to participate in this survey.

To access the survey click here

To view the Code of Practice for the Research click here.

The survey is broken down into several sections

1 Demographics
2 Server Demographics
3 Database Architecture, Design  and Development
4 Database Technical Practices
5 Data and Database Security
6 Change Management
7 Data Management
8 Frameworks
9 Storage
10 Cloud
11 Organizational Culture
12 Application Centric
13 Best Practice

If you would like to read a little more about my research the details are at http://sqltoolkit.co.uk/index.htm

PASS SQL Saturday #194

SQL Saturday http://www.sqlsaturday.com/ comes to Exeter in 2013. This event will be held on 8th and 9th March 2013 at Jurys Inn Hotel Exeter. 

On Friday 8th March the pre-conference training day sessions are:
Dave Ballantyne and Dave Morrison - TSQL and internals
Christian Bolton and Gavin Payne - Mission critical data platforms on Windows Server 2012
Denny Cherry - SQL Server Security
André Kamman - Powershell 3.0 for SQL Server Administrators and Developers
Mladen Prajdić - From SQL Traces to Extended Events - The next big switch.

On 9th March is SQL Saturday. Approximately 30 training sessions over 4 trackscovering all areas of working with SQL Server. For more details
http://sqlsouthwest.co.uk/sqlsaturday.htm 

BI Semantic Model (BISM)

The term BI Semanitc model has been bounded around a lot over the last year. I have come across a few really good articles which explain this in more depth.  The definition is that the  BI Semantic Model is one model for all end user experiences.

Here are my a few of articles that clearly explain this term with a really helpful diagram. 

Updated: Analysis Services – Roadmap for SQL Server Code Name “Denali” and Beyond
http://blogs.technet.com/b/dataplatforminsider/archive/2010/11/12/analysis-services-roadmap-for-sql-server-denali-and-beyond.aspx

So, what is the BI Semantic Model?
http://cwebbbi.wordpress.com/2012/02/14/so-what-is-the-bi-semantic-model/

Understanding the SQL Server 2012 BI Semantic Model (BISM)
http://www.mssqltips.com/sqlservertip/2818/understanding-the-sql-server-2012-bi-semantic-model-bism/

SQLBits XI Announced

The SQLBits Committee announced the dates of the next SQLBits conference.  SQLBits XI will be taking place on May 2nd-4th 2013 at the East Midlands Conference Centre in Nottingham, UK. More details will be here  http://www.sqlbits.com/ over the next few weeks.

Pass Summit 2012: Polybase What, Why, How.

[The original material links in this article are no longer available on the web and have been removed.] This was a talk in 2012 that I attended.]

This session was very interesting and shared the technical workings of Polybase. It was delivered by David DeWitt, Director at Microsoft from work in the Gray Systems Lab.  

He set the scene explaining the Hadoop ecosystem.

Then defined the main components.

This is a two universe world of both structured and unstructured data. The 2 alternative solutions are Sqoop which has limitations and Polybase which is a superior alternative.


 As stated on the Gray Systems Lab site "the goal of the Polybase project is to allow SQL Server PDW users to execute queries against data stored in Hadoop, specifically the Hadoop distributed file system (HDFS). Polybase is agnostic on both the type of the Hadoop cluster (Linux or Windows)"




The sessions discussed the approach and drawbacks of each method. There are several phases of delivery planned.

• Phase 1: in PDW next year
• Phase 2: working on
• Phase 3: thinking about

 David DeWitt ended the presentation with

• The world has changed
• Map reduce really is not the right tool
• Polybase for PDW is a first step.

The Data Lifecycle: Turning Data Into Business Value

The second PASS summit keynote was delivered by Quentin Clark, Corporate Vice President, Data Platform Group. The keynote took a journey through the data lifecycle. The data lifecycle being broken down into 5 areas.

• Collaborate
• Operationalize
• Manage
• Discover and refine
• Visualize

The lifecycle starts with managing all the data. Combining any data wherever it lives, be it relational and non-relational, cloud & on-premises and model architecture. Then from that data finding all the relevant information and ways to unlock the value of the data with models and analysis. After which creating visualizations of the data and allowing collaboration on the data. The data requires governance and control,  transforming insights into repeatable business process. Self service is the key element being delivered by the new tools.  The demos showed many of the available tools including Data Explorer http://www.microsoft.com/en-us/sqlazurelabs/labs/dataexplorer.aspx and how AlwaysOn allows you to add Azure as an availability Replica in SSMS which adds scalability.

The session ended with the message that Microsoft are providing a complete platform solution to let you find what the data is trying to say and embracing the new value of data.

Pass Summit 2012 Day One Keynote: Accelerate Insight on any Data

Today is the official start to the 2012 PASS summit. The PASS Summit 2012 Day One Keynote session was delivered by Microsoft corporate vice president of the Data Platform Group (DPG) Ted Kummert. The keynote was packed with announcements on changes in the database engines.

Service Pack 1

The first announcement made at PASS is the release of SQL Server 2012 SP1. Some of the new features include

• Cross-Cluster Migration of AlwaysOn Availability Groups for OS Upgrade
• Selective XML Index to improve performance
• DBCC SHOW_STATISTICS works with SELECT permission
• New dynamic management function which returns statistics properties (sys.dm_db_stats_properties)
• Express now comes with the complete SQL Server Management Studio (SSMS)
• SlipStream Full Installation
• Management Object Support Added for Resource Governor DDL

To read more read what's new in SQL Server 2012 SP1 http://bit.ly/YKmft4

Ted Kummert continued saying the world of data is changing with superabundant supplies. It is approaching a tipping point with volume, variety, velocity, hardware innovation, software innovation and cloud.  There is also a change in architectural assumptions with big data providung new insights with new sources of data.  There is the need to accelerate business process and insight, manage any data, any size, anywhere and enable pervasive insight.

Hekaton

The second announcement is project code name ‘Hekaton’ to accelerate in-memory for business process and insight. Project Hekaton brings an in-memory engine process to the transactional world. After converting tables to then in-memory engine there is almost a 10x performance increase. It allows recompiles of stored procedures so they will run in memory,  which could also offer a 30 times performance increase.
This is in the next release of sql.
The Hekaton AMR tool is designed to help identify hot spots in the database application and provide assistance to migrate things such as tables and stored procedures.

HDinsight

For Non-relational data Microsoft has released to CTP HDinsight Server. Microsoft’s Apache Hadoop based solutions for Windows Server and Windows Azure. http://www.microsoft.com/en-us/download/details.aspx?id=35397

SQL Server 2012 Parallel Data Warehouse
Microsoft announced the new SQL Server 2012 Parallel Data Warehouse. Queries that previously took 20 minutes to run only take 20 seconds. It offers an up to 50x performance gain with an optimized architecture. Due out in H1 2013.

Polybase

Polybase is a breakthrough in data processing. It integrates Hadoop data and relational data to allow combined TSQL queries with an optimized architecture. It will allow future expansion to other data sets. Polybase will unify the relational and non-relational world. Built for big data and coming in H1 2013.

Microsoft’s strategy is to design an Enterprise platform for  integrating data. We have to learn one new concept and that is about joining up data coming from multiple sources.

Other

Additional annoucements mentioned

• Clustered column store index with updatable tables
• Power View and PowerPivot fully integrated in Excel, now a complete BI tool
• Fully interactive maps inside Excel
• DAX queries on top of molap cubes

In conclusion data is bringing about lots of changes which adds richness  to the solutions.

PASS is running the first ever Business Intelligence conference in spring 2013 in Chicago.

Day 3 PASS SQL Rally Nordic

Day 3 launched straight into sessions. The session designing Hybrid Systems for the Enterprise by Buck Woody was based on the ebook  Building Hybrid Applications in the Cloud on Windows Azure http://www.tinyurl.com/9ffms3t . The session provided an excellent guide for creating a cloud architecture. The session finished quoting Plato “The beginning is the most important part of the work.”

The next couple of sessions were very useful, exploring under the hood with the sessions using statistics and relational algebra. The sessions were Optimizing column stores with statistical analysis by Thomas Kejser and the SQL Server 2012 Query Optimiser by Conor Cunningham.

Column store was originally written about in papers in the 1970’s and has only now been incorporated into SQL Server. Mathematical techniques were discussed, low cardinality, correlation of columns, entropy, mutual information leading to the big unsolved question of computer science  P=NP. More information is here http://blog.kejser.org/2012/07/27/what-is-the-best-sort-order-for-a-column-store/

The Query Optimiser session gave a flying visit to how the optimiser works. The optimizer finds a good plan rather than optimal plan as it may take to days to find the absolute optimal plan. SQL Server does a good job at selecting a good plan. The relational algebra trees were explained from the basics to logical , physical  tree concepts answering the question what is a query?  More details on previous talks on this subject matter http://sqlbits.com/Speakers/Conor_Cunningham

The last 2 sessions I attended covered Windows 2012 Infrastructure for SQL Server and Cloud Ready Data Services.

All in all an excellent conference with great sessions and lots of exchanging SQL views with like-minded people.

Day 2 PASS SQL Rally Nordic

The main conference started with a keynote delivered by Kamal Hathi BI on Big Data. This started with a historical look at database\ data progression. With social and web analytics and live data fields, how do we predict future outcomes? Utilising advanced analytics on big data this can be visually displayed with the creation of a storyboard connecting the many data sets such as SQL Server, Hadoop, Excel etc. with the Microsoft toolset. There is no one single tool but a collection of tools for the job.

Other sessions I attended were from the DBA track which covered extended events, clustering and memory management for SQL Server 2012.

An interesting session shared technical information on building highly scalable and available cloud applications. Key consideration for the architectural design

• Data Warehousing is not a good fit for the cloud
• Scale out not Scale up
• Everything has a limit
• Design for failure
• Design for continuity
• Optimise for density

The key to a successful cloud, distributed computing system, is all in the architectural design. A phase from the session “Telemetry is Life” left me thinking about patterns and practices.  The end of another  great day.

PASS SQL Rally Nordic 2012 Day 1

The PASS SQL Rally Nordic from October 1-3 2012 in Copenhagen started with 3 pre-conference seminars. I attended a session covering the Microsoft Business Intelligence (BI) Presentation layer taken by Stacia Misner. This excellent session covered the theory of the BI Maturity Model from TDWI by Wayne Eckerson, a roadmap to analytical completion and collaborative decision making. Then this was mapped to the BI stack of SharePoint, SQL Server Reporting Services, PowerView, PowerPivot, Performance Point and Excel.

Reading Material from the session

BI Maturity Model from TDWI by Wayne Eckerson
http://download.101com.com/tdwi/Poster/TDWI_BI_Maturity_Model_Poster_2005.pdf

Book on Competing on Analytics: The New Science of Winning  Thomas H. Davenport and Jeanne G. Harris

24 hours of PASS

24 hours of PASS provided 24 back to back hours of free SQL Server Training. The event ran from Thursday 20 September 2012 until Friday 21 September 2012. There were some amazing sessions again http://www.sqlpass.org/24hours/fall2012/ . This event gave you a glimpse into the level of forthcoming technical content to be covered at the PASS Summit 2012.

I was very interested in the session delivered by Denny Lee on the Introduction to Microsoft’s Big Data Platform and Hadoop Primer. He defined Big Data as 4V's volume, velocity, variability and variety utilising techniques and technologies that make handling big data at extreme scale economical. Big Data is allowing us to ask new sets of questions. He discussed Scale up and Scale out commoditized distribution.

Hadoop

 

He moved onto discussing what is Hadoop explaining that a lot of data is machine generated these days and the data is loaded first then modelled after.  The infrastructure allows automatic distribution and replication across nodes. Hadoop was based on the Yahoo Nutch project in 2003 and renamed to Hadoop in 2005. A reference book to look at is Tom White's Hadoop: The Definitive Guide.

To find out more about Microsoft's product  https://www.hadooponazure.com/ .

The Multi Facets of a Data Scientist

The Data Scientist role is an interesting evolution in the BI and Data Mining filed.  This role is the new name for a quant and an e-branding of statistical literacy as "data science". There are a few interesting quotes I have seen in the last few weeks defining data scientists which I thought were worth sharing.

•    Defines a Data Scientist as a person with mathematical and statistical skills, an investigative mind, an understanding of computer languages like C++ and Java and ability to write code. http://www.forbes.com/sites/tomgroenfeldt/2012/01/17/big-data-needs-data-scientists-or-quants-or-excel-jockeys/
•    A data scientist is somebody who is inquisitive, who can stare at data and spot trends. It's almost like a Renaissance individual who really wants to learn and bring change to an organization. http://www-01.ibm.com/software/data/infosphere/data-scientist
•    A data scientist includes not only crunching numbers, but also visualizing the results. http://www.technologyreview.com/view/425561/big-data-means-business-needs-mathematicians/
•    The data scientists look for patterns as branches and tributaries join and pull away from the “river.” http://www.forbes.com/sites/danwoods/2012/07/23/what-is-a-data-scientist-tom-wheeler-of-clickfox/2/
•    A new class of engineer, the "data scientist," whose job it is to perform the sophisticated mathematical gymnastics required to extract actionable information from this mass of numbers. http://www.technologyreview.com/view/425561/big-data-means-business-needs-mathematicians/

An article in computing http://www.computing.co.uk/ctg/analysis/2201583/computing-research-the-power-of-data-science explained the difference between the BI specialist and Data Scientist.

“ BI specialists understand data and analytics, their focus tends to be on the technical aspects such as implementing the software and controlling how data is stored within the system….. A data scientist is generally less concerned with the technical nuts and bolts and more interested in the analytical side, revealing the messages that lie hidden in the data and uncovering insights that can deliver immediate competitive advantage”

Data Scientists have good mathematical and communication skills, who will be analytic innovators through their ability to gain insight from big data. A recent study from EMC shared further insight on this new field http://practicalanalytics.files.wordpress.com/2012/01/datascientistinfographic.jpg

Database Landscape

Came across these excellant 2 diagrams from 2 very interesting blog posts.

“NewSQL-as-a-Service” Cloud Database

This diagram  shows where all the main database engines fit into the wider data management landscape.  Taken from http://xeround.com/blog/2011/04/newsql-cloud-database-as-a-service although the original article NoSQL, NewSQL and Beyond: The answer to SPRAINed relational databases is by Matthew Aslett which is part of the The 451 Group’s new long format report on emerging database alternatives, NoSQL, NewSQL and Beyond.
http://blogs.the451group.com/information_management/2011/04/15/nosql-newsql-and-beyond

Don’t Become a One-trick Architect
This article by Thomas Kejzer gives an historical look at architectures and data http://blog.kejser.org/2011/12/08/dont-become-a-one-trick-architect/#more-377
This is the evolutionary tree of life for data storage engines

SQL Server 2012 Install by Configuration File

Automating a SQL Server installation for unattended installation creates a consistent deliverable for a SQL Server build.

To create the configuration file run through the GUI installation selecting your predefined defaults and prior to clicking Install collect the configuration file from the specified path listed on the Ready to Install page e.g.  C:\Program Files\Microsoft SQL Server\110\Setup Bootstrap\Log\20120619_140314\ConfigurationFile.ini

The configuration file will need additional configuration details for unattended use as sensitive information is not included.

•  IACCEPTSQLSERVERLICENSETERMS is required to acknowledge acceptance of the license terms. 

• The QUIET and QUIETSIMPLE settings are required to allow the Setup to display progress only, without any user interaction.

A few parameter may need updating between server builds.

Security

The other sensitive information which is not written to the configuration file are the passwords. These can be passed into the installation configuration at run time using

Setup.exe /SQLSVCPASSWORD="************" /AGTSVCPASSWORD="************" /ISSVCPASSWORD="************" /SAPWD="************"
/ConfigurationFile=DefaultConfigurationFile.INI

The Service and SQL account parameters are:

SQLSVCPASSWORD: Account for SQL Server service: Domain\User
AGTSVCPASSWORD: Agent account name: Domain\User
ISSVCPASSWORD:  Account for Integration Services: Domain\User
RSSVCPASSWORD:  Account for Reporting Services: Domain\User
ASSVCPASSWORD: Account for Analysis Services: Domain\User
SAPWD:  Specifies the password for the SQL Server sa account.

The Microsoft article that describes the install of SQL Server 2012 using a configuration file can be found here http://msdn.microsoft.com/en-us/library/dd239405.aspx. A full list of parameters are stated here http://technet.microsoft.com/en-us/library/ms144259

Performance Dashboard Reports

The new custom  Performance Dashboard Reports for SQL Server 2012  can be obtained from http://www.microsoft.com/en-gb/download/details.aspx?id=29063

As with the earlier versions of these reports they allow a database administrator to quickly identify whether there is a current bottleneck on the system, and capture additional diagnostic data that can help resolve some common problems such as:

• Common performance problems that can be indeitifed through the dashboard are CPU bottlenecks (and what queries are consuming the most CPU)
• IO bottlenecks (and what queries are performing the most IO)
• Index recommendations generated by the query optimizer (missing indexes)
• Blocking
• Latch contention

Deployment

There are 3  parts to the deployment process.

1.  Run the run installer. The dashboard automatic install path places the files in C:\Program Files (x86)\Microsoft SQL Server\110\Tools\Performance Dashboard.
2. In the Performance Dashboard folder execute the setup.sql script.
3. Test the deployment by opening a report. Open the report by selecting the server instance in SQL Server Maangement Studio (SSMS) and in the menu options select Reports, then; Custom reports and navigate to the install path C:\Program Files\Microsoft SQL Server\110\Tools\Performance Dashboard. Then select  performance_dashboard_main.rdl and Click open and select Run.

The Dichotomy of Database Models

There has been significant debate on ACID and BASE and which is the way forward. A summary of the two models is described here.

ACID model

The relational database model for transactions are based on ACID properties, the term coined by Haerder & Reuter in 1983. This guarantees reliability, consistency and recoverability. It has four properties:

Atomicity
It is an ‘all or nothing’ approach. The transaction is performed in its entirety or not performed at all. Therefore if any part of the transaction fails it all fails and no change is made.

Consistency
The database transforms from one consistent state to another consistent state. The database cannot be left in an inconsistent state.

Isolation
Transactions execute independently of each other. This ensures that even if concurrent executions of transactions happen they are left in a state as if the transactions have executed serially. Partial and incomplete transactions should not be visible to other transactions.

Durability
Successful completed transactions are committed into the database and stored permanently. They persist after restarts.

New Patterns – CAP theorem

The concepts and patterns discussed in the paper ‘NoSQL Databases’ by Christof Strauch demonstrates the new concept to reduce complexity. The CAP-theorem was coined by Eric Brewer in 2000. It has three guarantees:

Consistency
The data is correct all of the time after the execution of an operation. A distributed system is typically considered to be consistent if, after an update operation, all readers see the updates in a shared data source.

Availability
It guarantees that data can be read and written all the time due to the architectural design and implementation.

Partition Tolerance
The system has the ability to continue to operate in the presence of a failure of part of the system.

BASE model
The concepts of NoSQL datastores use the  BASE pattern.  “The BASE approach according to Brewer forfeits the ACID properties of consistency and isolation in favour of “availability, graceful degradation, and performance” Strauch.

The acronym BASE is composed of the following characteristics:

Basic Availability
It uses replication techniques to reduce data unavailably. It is highly distributed and uses sharding and partitioning of data within its storage model.

Soft State
This allows data to be inconsistent and assumes that it is managed by the developer and not the database.

Eventual Consistency
At some point in the future the data will reach a consistent state. It is unknown when that will happen.

Brewer contrasts ACID with BASE  Design Patterns considering the two concepts as a complementing spectrum.

 

NoSQL Databases article page 32 by Christof Strauch

In conclusion we have two complementary design patterns for a universe where both structured and unstructured data reside.

Cube Synchronisation Scale Out Methods

There are 4 Analysis Services Synchronisation Methods

• Analysis Services Synch Method
• Backup/Restore Database
• Attach/Detach Database
• Robocopy Method

You can read more after the Querying and Processing cubes for scale out section

http://sqlcat.com/sqlcat/b/technicalnotes/archive/2008/03/16/analysis-services-synchronization-best-practices.aspx

Synchronize Analysis Services Databases
http://msdn.microsoft.com/en-us/library/ms174928.aspx

Other useful articles for scaling out Analysis Services are

REAL PRACTICES: Performance Scaling Microsoft SQL Server 2008 Analysis Services at Microsoft adCenter
http://sqlcat.com/sqlcat/b/whitepapers/archive/2011/03/14/real-practices-performance-scaling-microsoft-sql-server-2008-analysis-services-at-microsoft-adcenter.aspx

Scale-Out Querying with Analysis Services
http://sqlcat.com/sqlcat/b/whitepapers/archive/2007/12/16/scale-out-querying-with-analysis-services.aspx (download white paper http://technet.microsoft.com/library/Cc966449 )

Analysis Services 2008 R2 Performance Guide
Includes
Design Patterns for Scalable Cubes
Tuning Query Performance
Tuning Processing Performance
Special Considerations
http://sqlcat.com/sqlcat/b/whitepapers/archive/2011/10/10/analysis-services-2008-r2-performance-guide.aspx

Database Conference Wish List

Database conferences are the place to share ideas and learn new technology.  They are the place to expand ones database horizons and dream. My wish list for conference attendance would have to be

Professional Conferences

SQLBits
UK Location
http://sqlbits.com/
Probably End September 2012

SQLRally Nordic
Copenhagen
http://www.sqlpass.org/sqlrally/2012/nordic/
October 1-3 2012

SQL Pass Summit
Seattle, WA
http://www.sqlpass.org/summit/2012/
November 6-9 2012

Research Conferences

To explore cutting edge ideas in the database world.

ICDE 29th IEEE International Conference on Data Engineering
Brisbane, Australia
http://www.icde2013.org/
April 8-11 2013

2013 ACM SIGMOD/PODS 
New York, USA
http://www.sigmod.org/2013/ 
June 22-27, 2013

39th International Conference on Very Large Data Bases
Rive del Garda, Trento
http://vldb.org/2013/
August 26-31, 2013

Data Conferences

Data and information management.  

Enterprise Data World 2012
Atlanta, Georgia
http://edw2012.wilshireconferences.com/index.cfm
April 29 - 3 May 2012

SQL Relay

I attended SQL Relay 2012 in Clifton, Bristol yesterday which brought together  SQL experts from the southwest. The location was fantastic at the Avon Gorge Hotel which overlooks Brunel’s suspension bridge. The al fresco dining for lunch in the glorious Sun added to the interesting SQL discussions.

There were overview sessions from Microsoft in the morning on SQL Server 2012 and Deep Dive sessions in the afternoon from SQL Server MVP's.

The morning sessions delivered by Microsoft by Morris Novello, SQL Server Product Manager and Andrew Fryer, Technical Evangelist discussed the explosion of data volumes, new innovations in the cloud and new development features within SQL Server 2012. The practical overview of SQL Server 2012 new features showed the breadth and depth of the changes to the product which will take SQL Server to a new level.

The afternoon MVP’s Tony Rogerson covered SANless high availability with Always On and Martin Bell covered Distributed Replay. Considering disk geometry added an interesting dimension to Always On and being able to mitigate risk against hardware consolidation and changes using distributed replay was insightful.

Data Warehouse Physical Architecture Options

Custom Data Warehouse Architecture
Scale not defined
Custom design
  
Fast Track Reference Architecture
Scale between 4 and 80TB
Rapid Deployment and Use reference configurations optimized for data warehousing.
http://www.microsoft.com/sqlserver/en/us/solutions-technologies/data-warehousing/fast-track.aspx

HP Business Data Warehouse Appliance
Optimised for up to 5TB
Highly tuned solution that integrates hardware and software.
http://www.microsoft.com/sqlserver/en/us/solutions-technologies/appliances/hp-bdw.aspx

Parallel Data Warehouse
Scale to 100s of terabytes
Parallel processing (MPP) architecture to gain scalable performance, flexibility, and hardware choices with the most comprehensive data warehouse solution available.
http://www.microsoft.com/sqlserver/en/us/solutions-technologies/data-warehousing/pdw.aspx

SQL Server 2012 Databix

Enriched SQL-Flavoured cereal to nourish you and your customers

With three new ingredients to fortify customers and partners!

Cloud Based Learning Routes for SQL Server 2012

The volume of free training available for SQL Server in the cloud is continually increasing. The best sites I have found are below.

TechNet Virtual Labs: SQL Server 2012 Virtual Labs
http://technet.microsoft.com/en-us/virtuallabs/hh859575
This allows you to evaluate and test Microsoft's products and technologies with a series of hands-on labs that you can complete in 90 minutes or less.

Microsoft Virtual Academy
http://www.microsoftvirtualacademy.com/Home.aspx
This is a training portal where you can learn at your own pace, focusing on Microsoft technologies, gain points and get recognition. There are tracks in SQL Server, SQL Azure, Clustering, Business Intelligence etc.

SQL Server 2008 Microsoft Certified Master (MCM) Readiness Videos
http://technet.microsoft.com/en-us/sqlserver/ff977043
These videos are for those wanting to pursue SQL Server 2008 Microsoft Certified Master certification. Each module establishes the level of depth and breadth required to achieve this certification with deep dive subject matter included. These are good watch even if you are not thinking of taking the MCM exams.

SQL Server Best Practices
http://technet.microsoft.com/en-us/sqlserver/bb671430
Technical White Papers provide real-world guidelines, expert tips, and guidance for SQL Server.

EBooks
http://www.red-gate.com/community/books/
Free PDF downloads of ebooks specializing in database and programming technologies.

SQLBits
http://www.sqlbits.com/
Excellant community conference videos of recorded sessions which are available to watch on demand. 

24 Hours of PASS
http://www.sqlpass.org/24hours/spring2012/
The 24 Hours of PASS event is a virtual stage of SQL Server experts with in-depth sessions on the hottest SQL Server topics over 24 consecutive (and free!) technical webcasts.

SQL Server 2012 Certifications

The article 'Move Your Career to the Cloud with the Microsoft Certified Solutions Expert certification' sets out details of the new certification.
http://blogs.technet.com/b/nraja/archive/2012/04/12/move-your-career-to-the-cloud-with-the-microsoft-certified-solutions-expert-certification.aspx

This new set of SQL Server certifications aims to provide a broader and deeper set of technology skills. The new certification is across technology solutions based on cutting-edge technologies for both On-premise or in the cloud solutions. The certification details are

Microsoft Certified Solutions Associate (MCSA)
http://www.microsoft.com/learning/en/us/certification/cert-sql-server-MCSA.aspx
Exam 461 Querying Microsoft SQL Server 2012
Exam 462 Administering a Microsoft SQL Server 2012 Database
Exam 463 Implementing Data Warehouses with Microsoft SQL Server 2012

• To transition your certification
• Exam 457 Part 1
• Exam 458 Part 2

Microsoft Certified Solutions Expert (MCSE) Data Platform
http://www.microsoft.com/learning/en/us/certification/cert-sql-server-data-platform.aspx?WT.mc_id=v3_MSL_social_MCSEDP

Exam 464 Developing Microsoft SQL Server 2012 Databases
Exam 465 Designing Database Solutions for SQL Server 2012

• To transition your certification
• MCITP: Database Administrator 2008
• Exam 457 Part 1
• Exam 458 Part 2
• Exam 459 Transition your MCITP: Database Administrator 2008 to MCSE: Data Platform

MCSE: Business Intelligence
Further details of this route are here:-
http://www.microsoft.com/learning/en/us/certification/cert-sql-server-business-intelligence.aspx

SQLBits X - UK SQL Server 2012 Technical Launch Event

SQLBits X,  the official UK SQL Server 2012 Technical Launch Event was held from 29th - 31st March 2012 and took place at the Novotel London West, London.

This was another amazing event with plenty of chance to absorb the vast array of technical offerings from all areas of Database Management, Development and Business Intelligence.  It was a privilege to be a helper at such an event.

The event consisted of the Training Day, Conference Day and Community Day on Saturday. In the evening there was a chance to network with the largest attendance of any SQLBits.

The SQLBits keynotes were delivered by Quentin Clark , the Corporate Vice President of Microsoft SQL Server Database Systems Group and Conor Cunningham, Principal Software Architect at Microsoft. Quentin Clark showcased the vast set of new tools and new features of SQL Server 2012 to help deal with the data explosion and migration to cloud services. Conor Cunningham demonstrated the sheer speed of the new ColumnStore index and its benefits to the Data Warehouse.  A great plus for the conference was the presence of the SQLCAT team and international speakers.

The takeaway from the conference is that there is a change in how and what needs to be managed and where the future lies for database services.   

SQL Server 2012 Virtual Launch

SQL Server 2012 Virtual Launch Event today March 7, 2012 starting at 8:00AM Pacific (-8:00 GMT), 4PM in the UK.

The keynotes from Microsoft Executives:

Ted Kummert, Corporate Vice President, Business Platform Division
Quentin Clark, Corporate Vice President, Database Systems Group.

The launch video without login 
http://istreampl-push.wm.llnwd.net/istreampl_push

The Launch Event Agenda sqlserverlaunch.com/WW/Agenda

There are over 30 sessions to learn about the new capabilities of SQL Server 2012 at your own pace.

Management of Database Systems

Discussions held at the Avon Information Management group meeting on 1 February.

My current working research questions were discussed.

To what extent are best practices and procedures utilised by the database community?
Is their adoption affected by the complex interactions that are an integral part of the management of Database Systems? 
Can the use of holistic methodologies contribute to improvement and innovation?


The Database System
The 4 main components in the system:- 

• Technical Conponents
• Cultural factors
• Paradigms
• Application Centric

Technical Layers 
The current technical system discussed broken down into sections including:-

• Architecture
• Access and Control
• Maintenance
• Resiliance and Conservation
• Data Artifacts
• Change
• Forecasting

Cultural Factors
Items relating to cultural  arena:-

• Ethics and Governance
• Companies perception vs DBA’s
• Organizational management structure
• Customer requirements
• Financial and IT progression
• Proactive vs Reactive
• Database community
• Vendor  vs Supplier  vs Competitors
• Change Managers vs Developers vs Service Managers
• Attitudes and beliefs of DBAs
• Market place changes from local to global
• Environmental considerations
• Government, Legal and Political issues
• Business Owner, Changes to the business model, Risk

Paradigms
A few major methodologies that are in use today:-

• Service Management
• Data Management
• Problem Management
• Forensic Management
• Agile Management
• Information Lifecycle Management
• Database as a Service

Additional paradigms to consider:-

• Service Orientated Architecture
• Capability Maturity Model

Application Centric
Taking the viewpoint that the application is at the centre of everything:-

• Structured vs Unstructured vs Semi Structured
• Transient vs Transactional vs Historical
• Big Data vs Scalability, Sharding
• Scale out vs Scale up
• Quality of Data
• Cloud vs On Premise
• Data Sets for Public consumption
• Database Engine changes vs Virtualization

Architectural Design issues: Scale Up and Scale Out

Scale up is about vertical scalability. This means increasing the capacity by adding more CPU and memory.

Scale out is about horizontal scalability. This means increasing the number of servers available.

Brent Ozar wrote a couple of really interesting articles

Scaling up or scaling out?
http://www.brentozar.com/archive/2011/02/scaling-up-or-scaling-out/
Scaling up of scaling out? part 2
http://www.brentozar.com/archive/2011/03/scaling-up-or-scaling-out-part-two/

Interesting analogy here
Scale Up or Scale Out your Data Problems? A Space Analogy
http://dennyglee.com/2012/01/24/scale-up-or-scale-out-your-data-problems-a-space-analogy/

2012 SQL Year of the Dragon

The year 2012 is associated in Chinese astrology with power and wisdom. With the release of SQL Server 2012 this year we will see the power of the product multiply with a vast array of new features. But what will the future of data and databases bring. There are many post around trying to predict where the database world will shift this year. Here are a few that took my interest

Michael Stonebraker thinks

• one size no longer fits all
• “big data” means three different things  (could mean big volume, velocity and too much variety of data sources)
• ACID [atomicity, consistency, isolation, durability] is still a really good idea
• Think memory. It’s the new disk

http://searchdatamanagement.techtarget.com/news/2240113325/Michael-Stonebraker-predicts-trouble-for-relational-databases-in-2012

Forrester Predictions for 2012 are that

• Enterprise Hadoop deployments will expand at a rapid clip
• In-memory analytics platforms will grow their footprint
• Graph databases will come into vogue

http://blogs.forrester.com/james_kobielus/11-12-19-the_year_ahead_in_big_data_big_cool_new_stuff_looms_large

Edd Dumbill lists the five big data predictions for 2012 as

• More powerful and expressive tools for analysis
• Streaming data processing
• Rise of data marketplaces
• Development of data science workflows and tools
• Increased understanding of and demand for visualization

http://radar.oreilly.com/2011/12/5-big-data-predictions-2012.html

Big Data, analytics get even bigger, hotter in 2012 as Chris Kanaracus writes

• Big Data is not going away
• In-memory processing will be the belle of the ball
• Will analytics-as-a-service take off?
• BI and analytics start showing up everywhere

http://www.computerworld.com/s/article/9222996/Big_Data_analytics_get_even_bigger_hotter_in_2012?taxonomyId=18&pageNumber=1

How wise will we become from all this additional data and knowledge?  Data will rule the day and it will be provided by which ever tool meets the requirements, whether it be SQL, NoSQL, Oracle, MySQL etc.