Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Showing posts with label Best Practice. Show all posts
Showing posts with label Best Practice. Show all posts

Monday, 9 August 2021

A Summer Retrospective: a bygone era


A few weeks in rural France is just the place to contemplate life and take a step back into a bygone era. An era where there are no phones, no internet and no television. Life can quite easily pass you by and you could go for weeks not speaking to a sole. The fruit on the trees ripen in the orchard, the birds waiting for the perfect moment to swoop and eat the fruit. The roads are mostly empty with the occasional car or logging lorry passing by. Cycling is heaven with the roads to yourself.

This rural area, 214 million years ago, had all life within 300 miles of Rochechouart wiped out when a meteorite, around one of the 15 largest ever to come crashing down on earth. The geological signs of he creator are still present today. This bygone era was also rife with conflict. From the last battle of Richard the 1st - the Lionheart, who laid siege to the Chateau of Chalus-Chabrol, located at the border between Aquitaine and the French kingdom, to the hideouts in the forest of the Maquis du Limousin, who were one of the largest groups of French resistance fighters in the Second World War. The village of Oradour-sur-Glane remains an empty ruin as a memorial for the massacre of its inhabitants.

In this backdrop, technology seems a lifetime away. I can't stress enough the tremendous benefits of taking a technological break for your mental health. You can dream and innovative without the interruption of everyday life.

The age of cloud computing, big data and the algorithm requires a 360-degree perspective. A socio-technical perspective is critical. Reflecting on the changes to the earth, made to this unique landscape from space, you realize that data is in the environment. It is not possible to be an expert in all areas as data is the environment. Data is history, is in the maps, is used in conflict resolution and is used for impact analysis. Data is completely inseparable from life and it drives life, not only business. The  choice of tools available help you navigate through data are vast.

The question is can one truly ever master the entirety of life. Data is life, the past, the present and the future. To truly be a master it requires collaboration, communication and control, as data weaves its interconnected complexity throughout life. A holistic view of this diverse scientific area is required to provide a sustainable future. There is no one best practice that can help navigate this web of graph vertices and edges.

To that end I summise that taking a technological break enables the mind to contemplate and blue sky thinking roam free. Happy Summer break. 

Thursday, 5 September 2019

The Mirage and Metamorphosis of Data and AI



I have just taken a break to rejuvenate my creative juices. It was a time to reflect and innovate. We are often so busy in our day to day lives we don't stop and reflect. I spent my time reading and catching up on bleeding edge technology. I am always fascinated to see what is coming next, what problems researchers are trying to address and how Data and AI could be utilised to benefit industry and the world around us.

The role I enjoy the most is as a Data and AI philosopher providing thought leadership. We are at an exciting time in history to witness and contribute to the mirage and metamorphosis of Data and AI. My explorations find exciting challenges in diversity and Data and AI at the centre of most things we want to achieve. Research is increasingly needed in industry to achieve business success due to the increasing complexity within industry and the world around us. We need to move away from agile for certain tasks to enable complexity to be understood and use systems thinking techniques.  My findings on the future mirage and metamorphosis of Data and AI are a complex interconnected world around Data and AI and mastering that complexity is the key to success.



  











References
The data and AI market landscape 2019: The next wave of hybrid emerges
https://www.zdnet.com/article/the-data-and-ai-market-landscape-2019-the-next-wave-of-hybrid-emerges/
Part I: A Turbulent Year: The 2019 Data & AI Landscape
https://mattturck.com/data2019/
Part II: Major Trends in the 2019 Data & AI Landscape
https://mattturck.com/2019trends/
Navigating AI hype in search of success, Oliver Pickup (Sunday Times 12 May 2019)
The real big-data problem and why only machine learning can fix it
https://siliconangle.com/2019/08/09/real-big-data-problem-machine-learning-can-fix-mitcdoiq-startupoftheweek/
Big Data is just Data
https://buckwoody.wordpress.com/2019/08/26/big-data-is-just-data/
Maximising the AI opportunity
https://info.microsoft.com/rs/157-GQE-382/images/UK-DIGTRNS-CNTNT-content-MGC0003240.pdf
The Data Ethics Framework principles
https://www.gov.uk/government/publications/data-ethics-framework/data-ethics-framework

Wednesday, 6 February 2019

Improved Microsoft Docs


A cool image from http://www.thinksinc.org/ about Microsoft Docs.

I was looking at the Microsoft Docs pages and its new design. I have found it is much easier to navigate which speeds up searching.

At the top of the page there are 3 helpful options 

  • Download SQL Server
  • Get an Azure VM with SQL Server
  • Download SQL Server Management Studio


Then the Microsoft SQL Documentation has 3 categories covering on premises and cloud.
  • SQL Server on Windows
  • SQL as an Azure Service
  • SQL Server on Linux
There are technology areas to drill down further.

Then a further collection of links to enable a deeper dive into the technology.

  • Design
  • Tools
  • Reference
  • Reporting
  • Data Analytics
  • AI and Machine Learning
I was looking for design documentation and the link takes you to a page with easy to select image and text.























Saturday, 28 July 2018

What is Best Practice?


Best practice is a pervasive term that means different things to different people. Best practice has been defined in various ways (Dembowski 2013; Wellstein & Kieser 2011; Sanwal 2008). Dani et al. (2006) stipulate “A best practice is simply a process or a methodology that represents the most effective way of achieving a specific objective”. Jarrar & Zairi (2000) state that the term best practice is often used within organizations to depict leadership and is recognised as the best way to achieve superior results. In the glossary of benchmarking terms (American Productivity and Quality Centre 1999) cited in (Jarrar & Zairi 2000, p.S734) best practices were defined “Those practices that have been shown to produce superior results; selected by a systematic process; and judged as exemplary, good, or successfully demonstrated. Best practices are then adapted to a particular organisation”. Many different situations require different best practices and with new technology evolving ‘best’ is a moving target (Jarrar & Zairi 2000). 

Markus (2011, p.4) argued that the cultures and practices that develop over time in organizations have changed to become “off-the-shelf” services labelled best practice standards, which organizations needed to adopt and understand. Markus argued the change from unique coded management ideas for handling packages to standard software with relentless upgrades requires knowledge development and standard practices.

Sanwell (2008) stated that the use of best practices are affected by certain beliefs:
  • Best practices help make decisions quickly in a complex uncertain world. 
  • Best practices are easier because they have been proven by other organizations who also operate with complex and uncertain elements. 
  • Management understanding of other organizations in the field are organizational specific. Best practices are often developed later and often already behind leading organizations. 
  • Value must be gained from best practices as other experts, consultants and vendors share them for current trends. 
  • Best practices can improve performance. 

Falconer (2010) argued to the contrary that best practice exacerbates failure:
Best practice is flawed because it acts as a placeholder for proper management practice, displacing accountability for effectiveness and fit. Best practice is flawed, further, because it supplants strategy, adopting solutions out of convenience or copying them reactively, and supplants innovation, allowing “the best we know about”, “the best we’ve come across”, or even “the best we’ve done before” to be adequate. Best practice considers the world predictable, and discounts the emergence of better, novel ideas(Falconer 2010, p.754)
Falconer thought that problem situations are being incorrectly handled due to best practices replacing analysis.

Sanwell (2008) pointed out that changing these best practices in the multidimensional world requires consideration of organizational culture and behaviour, organization processes and organizational systems. As Gonnering stated,
“Best Practices” can serve as a beginning but adaptation will most likely be necessary. Outcome is an emergent property, and the organization that has taken the time to learn the methodology of improvement will reap the benefits. The “continuous” in “continuous quality improvement” depends upon rapid-cycle, small-scale serial innovation and not a static and dogmatic adherence to past processes.” (2011, p.100) 

Gonnering argued that complex problems using best practices failed to have positive outcomes and forced the complex systems to become chaotic. Bretschneider et al. (2004) highlighted three important characteristics of best practice: a comparative process, with action, and linked to an outcome or goal. Nattermann (2000) suggested best practice might be the most widely used management tool in business and important for improving operational efficiency, but for strategic decision making, best practices might not be the best way forward to increase profit margins. Best practices management could be used to benchmark performance, with certain benchmarks being required to demonstrate best practices.

The core or classic best practices utilised within the database community have been developed through the sharing of knowledge, experience and actual outcomes across the sector. The improvement of these best practices were raised by Gratton & Ghoshal (2005) with the term “a signature process”, a process that envelops the company’s character and idiosyncratic nature. This signature process could advance the company although it required careful adaptation and alignment to business goals to succeed. However the allure of classic best practices that were clear, logical and easy to understand were the ones shared within the database community, the body of knowledge often yielding optimal results (Tucker et al. 2007). Some best practices were tightly coupled with their organizations and inseparable from the context (Becker 2004).

Jarrar & Zairi (2000) identified three types of best practice: proven best practice across organizations, good practice techniques for an organization, and unproven good ideas based on intuition. There were drawbacks with unproven ideas that could be a matter of luck and the lack of information to reduce the risk, lack of situational context, application criteria or success measure (Falconer 2011). This serendipitous discovery could lead to ease of deployment and innovation.

The Cynefin framework (Snowden & Boone 2007) classified and ordered simple systems in the domain of best practices. In an earlier paper in the chaos domain Kurtz & Snowden (2003) argued that applying best practices probably caused the chaos in the first place. They argued that different contexts use different management responses and that there are different tools for the management of complex contexts. The best practices domain is based on cause and effect relationships that have simple contexts, often within areas that do not change frequently.

Wagner and Newell (2011, p.400) stated that “The best way of operationalizing a process in one context and at one point in time may be different in another context and time”. They contended that there is no such thing as best practice, as knowledge is created by engagement in a practice. Practice is always changing and emergent with inconsistencies in the same practice, with best practice being defined locally. 

Wagner and Newell (2011, p.401) suggested a move to negotiated practice with a cooperative approach to best practice adoption. Their aim was to smooth out complex implementation through compromise. They concluded that highlighting problems with identifying best practice (due to it being an interactive process based on learning through implementation with information systems) sometimes required customisation to work well. This approach was also adopted by Avgerou & Land (1992) with their notion of ‘appropriate’ context specific practice, where information systems innovation looked for “best practice, or suitable new organizational form for the information age”  (Avgerou 2011, p.650).

Avgerou drew together organizational and information systems to develop a framework which had one key tenet of a knowledge management system or a best practice solution to help address static and commoditized technology.

Best practices and procedures were continually developed by database software providers (e.g. Microsoft, Oracle and MongoDB) to enable the management of database systems to be carried out to the highest standards. The procedures were based on formal rules the business world defined which were sometimes called standard operating procedures (Becker 2004). Best practices were defined by the software providers as exemplary tested designs for certain configurations or ways of doing things. They were multi-faceted and resided in varying layers from architectural design, through development, to operational management.

The management of database systems utilizes best practices and procedures provided by software providers and often industry best practices shared by the community. McGregor (2007) argued that this rarely leads to great customer service. McGregor’s (2007) idea that “Next Practice” was the future of continually analysing and looking for positive quality products and service in other organizations, would bring ideas and innovation to improve the business. There was an aspiration to improve database management and improve business processes to provide good quality service when managing IT projects and database systems. Best practices might not however be the best solution. Sanwell (2008) raised some key issues with using processes and strategies created by other organizations, and did not believe that following these would create a better organization or bring about improvement.

Within database systems there are various types of practices and procedures that need to be incorporated within change processes. Savage (2014, p.17) stated Stonebraker thought “in memory” database engines will take over online transactional processing systems (OLTP). Savage (2014, p.16) shared Stonebraker’s views on the database world, that it could be divided into three types: OLTP, data warehouses and everything else (Hadoop, graph databases). This was likely to mean three or more database management and best practices models were required.

Best practices operate at different levels within the sphere of database management. There are technology best practices which deal with specific tasks for deployment of databases onto servers or into the cloud; and management best practices which relate to higher level functions and overall processes. In addition there are best practices which are defined by software vendors for their own products.  As technology and management change, in the world market, and more is understood about certain areas, best practices change. Thus best practices are replaced with new best practices. The large collection of best practices created are likely to be defined and owned by a multitude of people. This can cause problems with conflicting best practices. Sometimes there is a mismatch between best practices and a compromise needs to be found where possible.
   
Best practices are intended to be useful for technical solutions to help people provide the required results. They aim to provide a useful guide on what management need to do to perform certain tasks. Best practices are sometimes adapted from vendor or industry defined best practices for nonstandard configurations or different business scenarios. However, sometimes communication is lacking between the management requirements, the vendors’ practices and the technology tasks. Different teams may each create best practice, in places where the technology overlaps, which are not shared. There are therefore limitations to the usage of best practices. The best practices presented are significantly different for ILTM, CMM and ILTIL. There are many different types of tasks from in depth technical ones to higher level models that combined can produce a well-managed database system. Each task, model or part of the database system will have its own best practice, which aims to achieve those reliable results. These best practices at different levels may, in practice, sometimes be in conflict. This discussion on best practice has shown there are many diverse views on the usability and definition of best practice. The working definition in my research (Holt, 2017) for best practice was: a recommended practice for carrying out actions for desirable outcomes, rather than always being the best way of doing something. The research best practice findings are in Holt et al. (2015) and the working cogs of best practice summaries the findings. 

American Productivity and Quality Centre. (1999). What is benchmarking. Retrieved from www.Apqc.org

Avgerou, C. (2011). Discources on innovation and development in information systems in developing countried research. In R. D. Galliers & W. L. Currie (Eds.), The Oxford Handbook of Management Information Systems (p. 650). Oxford: Oxford University Press.
Avgerou, C., & Land, F. (1992). Examining the appropriateness of information technology. In S. Odedra & M. Bhatnagar (Eds.), Social Implications of computers in developing countries (pp. 26–42). New Delhi: Tata McGraw-Hill.
Becker, M. C. (2004). Organizational routines: a review of the literature. Industrial and Corporate Change, 13(4), 643–678. https://doi.org/10.1093/icc/dth026
Bretschneider, S. (2004). “Best Practices” Research: A Methodological Guide for the Perplexed. Journal of Public Administration Research and Theory, 15(2), 307–323. https://doi.org/10.1093/jopart/mui017
Dani, S., Harding, J. a, Case, K., Young, R. I. M., Cochrane, S., Gao, J., & Baxter, D. (2006). A methodology for best practice knowledge management. Proceedings of the Institution of Mechanical Engineers, Part B: Journal of Engineering Manufacture, 220(10), 1717–1728. https://doi.org/10.1243/09544054JEM651
Dembowski, F. L. (2013). The Roles of Benchmarking , Best Practices & Innovation in Organizational Effectiveness. International Journal of Organizational Innovation, 5(3), 6–20.
Erica Wagner, & Newell, S. (2011). Changing the story surrounding enterprise systems to improve our understanding of what makes erp work in organizations. In R. D. Galliers & W. L. Currie (Eds.), The Oxford Handbook of Management Information Systems (p. 401). Oxford: Oxford University Press.
Falconer, J. (2010). “Best Practice” as Worst Practice : Broken Metaphor , Nude Emperor. Proceedings of the European Conference on Intellectual Capital, 754–762.
Falconer, J. (2011). Knowledge as Cheating : A Metaphorical Analysis of the Concept of “Best Practice.” Systems Research and Behavioral Science, 180, 170–181. https://doi.org/10.1002/sres
Gonnering, R. S. (2011). The Seductive Allure Of “Best Practices”: Improved Outcome Is A Delicate Dance Between Structure And Process. E-CO, 13(4), 94–101.
Gratton, L., & Ghoshal, S. (2005). Beyond Best Practice. MITSloan Management Review, 46(3).
Holt, V. et al. (2015) ‘The usage of best practices and procedures in the database community’, Information Systems, 49. doi: 10.1016/j.is.2014.12.004.
Holt, V. (2017) A Study into Best Practices and Procedures used in the Management of Database Systems. The Open University. Available at: http://oro.open.ac.uk/id/eprint/50950.
Jarrar, Y. F., & Zairi, M. (2000). Best practice transfer for future competitiveness: A study of best practices. Total Quality Management, 11(4–6), 734–740. https://doi.org/10.1080/09544120050008147
Kurtz, C. F., & Snowden, D. J. (2003). The new dynamics of strategy : Sense-making in a complex and complicated world. IBM Systems Journal, 42(3). https://doi.org/10.1147/sj.423.0462
Markus, M. L. (2011). Historical Reflections on the Practice of Information Management and Implications for the field of MIS. In R. D. Galliers & W. L. Currie (Eds.), The Oxford Handbook of Management Information Systems (pp. 3–15). Oxford: Oxford University Press. https://doi.org/http://dx.doi.org/10.1093/oxfordhb/9780199580583.003.0002
McGregor, M. (2007). When Best Practice is Just Not Good Enough Why and How You Need to be Better than the Best. BPTrends, (July), 1–2.
Nattermann, P. M. (2000). Best practice does not equal best strategy. The McKinsey Quarterly, 2.
Sanwal, A. (2008). The Myth of Best Practices. Journal of Corporate Accounting & Finance, 19(5), 51–60. https://doi.org/10.1002/jcaf
Savage, N. (2014). The Power of Memory. Communications of the ACM, 57(9), 15–17. https://doi.org/10.1145/2641229
Snowden, D. J., & Boone, M. E. (2007). A Leader’s Framework for Decision Making. Harvard Business Review, 85(11), 68–76.
Tucker, A. L., Nembhard, I. M., & Edmondson, A. C. (2007). Implementing New Practices: An Empirical Study of Organizational Learning in Hospital Intensive Care Units. Management Science, 53(6), 894–907. https://doi.org/10.1287/mnsc.1060.0692
Wellstein, B., & Kieser,  a. (2011). Trading “best practices”--a good practice? Industrial and Corporate Change, 20(3), 683–719. https://doi.org/10.1093/icc/dtr011





Monday, 30 April 2018

Machine Learning Algorithm Cheat Sheet

Another machine learning cheat sheet to help you choose your algorithm. The cheat sheet is designed for beginner data scientists and analysts.


The types of learning.


Saturday, 14 April 2018

Ph.D Graduation

“A story has no beginning or end: arbitrarily one chooses that moment of experience from which to look back or from which to look ahead.”
― Graham Greene, The End of the Affair

After 7 years of hard work, bringing industry and research together, I was excited to attend my Ph.D graduation. What an awesome and humbling day. Words can't express how it felt as a Ph.D graduate, with a Doctor of Philosophy, to sit on the stage along side the university academic staff. It is something I will never forget.

Now it is time to utilize my research skills gained throughout the Ph.D and begin something new. My aspirations in the academic field, are to write many  papers, share my research findings and to become a research fellow. 

Sunday, 1 April 2018

Literature Map


When you start any research project, you need to set the research in the context of the current literature. This will establish a framework for the importance of the study. This document was the starting place for organizing the literature of interest in my research.

Thesis Title: A Study in Best Practices and Procedures for the Management of Database Systems



Monday, 13 November 2017

A Guide to Complexity of Database Systems


The Phd research I undertook examined the complexity of database systems. A summary of the findings are provided

In the turbulent fast moving field of database systems, complexity is found everywhere. The volume, variety and velocity of data is continually expanding as well as the accessibility and realization that businesses have a wealth of untapped data that can be democratised. Not only this, and changes in new technology, but also with the shift in business markets, organisational changes, knowledge required by operating staff and numerous stakeholders, adds to the complexity. Many of these complexities have been discussed in the Claremont and Beckman Reports (Agrawal et al. 2009; Abadi et al. 2016)

This guide takes a 360 degree view of the situation through a systems thinking lens, providing synthesis between the cross disciplinary fields. To be able to explain what complexity is shapes our understanding of the situation and a basic visualisation of this is shared through the use of a graph. The usage of graphs as visual representation are discussed with the presentation of the graph metrics leading to the CODEX, a blueprint for the management of database systems. The CODEX could enable transformation of the management of database systems so that actionable insight can be achieved.

Monday, 12 December 2016

Data past, data present and data future

Database and data management practices are changing and new practices need to be adaptive and agile. Here are a few data thoughts for data past, data present and data future.



















Data of Christmas Past
  •  Reports and dashboards have become standard place within the business. 
  •  Big data is just the new look of data.
  • DevOps has become mainstream.
Data of Christmas Present
  • There is an increasing plethora of database architecture designs to choose from which means selecting the right design and database engine for the right job is harder than before.
  • Business is driving the need for data. 
  • Best practice delivery is hard in a fast changing environment. 
  • Complexity is increasing in a diverse landscape.
Data of Christmas Yet to Come 
  • The face of database administration is changing with multiple of types of engines, tools, applications and cloud offerings. 
  •  It is necessary to have broad range of database knowledge to ensure best practice configurations are deployed for the plethora of tools.  
  • Predicative analytics are becoming critical for business. 
  • Deep learning utilizes machine learning to model data at high levels of abstraction which will transform how we live.
  • Research is starting to become embedded in industry with the need to drive the next innovation.