Cloud Migration Strategy

I had to share this article because of the picture. This article looks at the merits of each migratory path to plan your journey. The options discussed are: 

• Lift and Shift
• Evolve - between lift and shift and a full rebuild
• Go Native - Cloud-native apps are designed for the cloud, so assume that the infrastructure they run on is inherently unreliable, but also controllable. They should be largely self-aware, self-scaling and self-healing. Most of the functionality is simply consumed rather than written into the code.

Azure Cosmos DB multi-model database

Azure Cosmos DB has to be one of my favorite databases due to the breadth of available database types, its choice of consistency models and elastic scale out.

An introduction can be read here.

A definition for each of these types of databases is given.

Key-value
A key-value pair (KVP) is a set of two linked data items: a key, which is a unique identifier for some item of data, and the value, which is either the data that is identified or a pointer to the location of that data. Key-value pairs are frequently used in lookup tables, hash tables and configuration files.
https://searchenterprisedesktop.techtarget.com/definition/key-value-pair

Column
A column-oriented DBMS (or columnar database management system) is a database management system (DBMS) that stores data tables by column rather than by row.
https://en.wikipedia.org/wiki/Column-oriented_DBMS

Document
Document stores, also called document-oriented database systems, are characterized by their schema-free organization of data.That means records do not need to have a uniform structure, i.e. different records may have different columns. The types of the values ​​of individual columns can be different for each record. Columns can have more than one value (arrays). Records can have a nested structure. E.g. MongoDB
https://db-engines.com/en/article/Document+Stores

Graph
A graph database, also called a graph-oriented database, is a type of NoSQL database that uses graph theory to store, map and query relationships. Every node in a graph database is defined by a unique identifier, a set of outgoing edges and/or incoming edges and a set of properties expressed as key/value pairs.
https://whatis.techtarget.com/definition/graph-database

The five consistency levels offer predictable low latency guarantees and multiple well-defined relaxed consistency models.

Consistency Levels and guarantees

Consistency Level	Guarantees
Strong	Linearizability. Reads are guaranteed to return the most recent version of an item.
Bounded Staleness	Consistent Prefix. Reads lag behind writes by at most k prefixes or t interval
Session	Consistent Prefix. Monotonic reads, monotonic writes, read-your-writes, write-follows-reads
Consistent Prefix	Updates returned are some prefix of all the updates, with no gaps
Eventual	Out of order reads

There is a useful capacity planer that looks at request units throughput per second, request unit consumption and the amount of data storage needed by your application.

Inspire 2018 - Intelligent Cloud and Edge

Inspire 2018 is in Las Vegas, Nevada, 15-19 July. This is the Microsoft premier event for partners. It has been interesting to see the discussions on Twitter during Inspire. These three quotes sum up a possible future state.

Without data, AI doesn't work. Data is the fuel for any AI service @TimNilmaa

It is our collective objective to democratize AI in every industry @Microsoft

The era of the intelligent edge has three defining characteristics AI, ubiquitous computing and people centred experiences @Microsoft

These three areas tie in with the Satya Nadella book 'Hit Refresh' that I have been reading. It is a truly extraordinary book that inspires you to innovate and use that innovation for a better future.

I found in my own research, people centered experiences, drive much of what, where and how things are achieved. It is these small differences in inputs,  that have the possibility to change the outputs in such variable ways. This leads to an expanding complex environment. But what if you could understand that complexity and adjust the tasks that need to be carried out in diverse cross disciplinary teams by building in AI and using a CODEX (Control Of Data EXpediently) model, when managing data and database systems. The result would stretch the breadth and depth, from on premises to cloud delivery, incorporating current and future states. A fully autonomous aid to managing data and database systems using the CODEX that is described my research should help gain that intelligent edge.

Microsoft MVP Award

I received my first Data Platform MVP award yesterday. What an honour it is to be a part of an amazing community. I am humbled by the enormity of the award and it is a privilege to be able to share my passion for data. I am listed here.

Databricks in Azure

Databricks is a big data unified analytics platform that harness the power of AI. It is built on top of Spark, serverless and is highly elastic cloud based. Azure Databricks is in preview currently. This new Azure service aims to accelerate innovation by enabling data science with a high-performance analytics platform that’s optimized for Azure. It has native integration with other Azure services such as Power BI, SQL Data Warehouse, Cosmos DB as well as from enterprise-grade Azure security, including Active Directory integration, compliance, and enterprise-grade SLAs. More information can be found in these two links

A technical overview of Azure Databricks
https://azure.microsoft.com/en-gb/blog/a-technical-overview-of-azure-databricks/

Introduction to Azure Databricks
https://channel9.msdn.com/Events/Connect/2017/T257

Databricks is a collaborative workspace.

Databricks have an ebook Simplifying Data Engineering to Accelerate Innovation which covers

• The three primary keys to better data engineering
• How to build and run faster and more reliable data pipelines
• How to reduce operational complexity and total cost of infrastructure ownership
• 5 examples of enterprises building reliable and highly performant data pipelines

Journey to the cloud

I have been reading a very interesting free ebook Enterprise Cloud Strategy by Barry Briggs and Eduardo Kassner. 

There are various types of modernisation discussed. 

It is based on Based on “Gartner Identifies Five Ways to Migrate Applications to the Cloud”, Gartner Inc., 2011. 

The overview of cloud migration principles discussed are summarised:

There is new a model that truly adopts cloud taking into account SaaS, PaaS and IaaS. This new architecture principle means considering the best placement of workloads, always looking at SaaS first.

This new way of thinking follows this placement decision tree which helps you make the correct decision on whether to use SaaS, PaaS, IaaS or private cloud.

On the data side, the paper talks about dividing your data into several categories related to risk. This is really about creating your own business data DNA. 'Data analytics' and 'BI and analytics'  cloud architectural blueprints are shared.

An interesting change for roles is the evolution of roles for the cloud. Microsoft suggest the data roles of Data Scientist and Information Architect emerge from the DBA and Statistician.

AWS Summit 2017 London

Yesterday was my first AWS Summit at ICC London, ExCeL. The event was much larger than I thought. On arrival I had to queue for over 25 minutes to get into ICC London. Then had to queue again to collect the registration badge, then had to go to another room to be security checked. 45 minutes later managed to get a seat in the keynote which had already started.




















There was a great demo showing robots working in a distribution centre at @OcadoTechnology using AI. The keynote raised the question of old world relational databases. Do we still need old world relational databases? They say not and MySQL, PostgreSQL and MariaDB are why customer are moving to Open database engines.

The Forces and market trends that are causing adoption of new services

Amazon has various database offerings.

• Amazon Aurora (a MySQL-compatible relation database engine that combines the speed and availability of high-end commercial databases)
• Amazon Relational Database Service (covers MySQL, Oracle, SQL Server, or PostgreSQL database in the cloud)
• Amazon DynamoDB (a highly scalable, fully managed NoSQL database service)
• Amazon Redshift (a fully managed, easily scalable petabyte-scale data warehouse service that works with your existing business intelligence tools)
• Amazon ElastiCache (a web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud.)
• AWS Database Migration Service

Migrating Data to and from AWS

When migrating databases to AWS you can backup your SQLServer database and restore the backup file to AWS RDS or EC2. The migration steps

There were various things to learn from the migration.














The Amazon Relational Database Service (Amazon RDS) covers

Amazon Aurora keeps 6 copies of the data, two copies in each availability zone to protect against availability zone failure.

There are various Analytics services such as

• Amazon Athena (is an interactive query service to analyze data in Amazon S3 using standard SQL. Athena is serverless)
• AWS Data Pipeline (is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premise data sources, at specified intervals)
• AWS Glue (is a fully managed ETL service that makes it easy to move data between your data stores)
• Amazon Kinesis (to collect, process, and analyse real-time streaming data) 

Artificial Intelligence service offerings are: 

• Amazon Lex
• Amazon Polly
• Amazon Rekognition
• Amazon Machine Learning
• AWS Deep Learning AMIs
• Apache MXNet on AWS

AWS was called the centre of gravity for AI.

Machine Learning capabilities explained

Azure Application Architecture Guide

I came across this useful guide. It presents a structured approach for designing applications on Azure that can scale, are resilient, and highly available. It is based on proven practices that Microsoft have learned from customer engagements.

Implementing Microsoft Azure Infrastructure Solutions

Here are a whole set of links to kick start your learning of Microsoft Azure services. 

Introduction video

Changes to computer thinking - Stephen Fry explains cloud computing 

Two Portals

• ASM Classic (Azure Service Manager) 
• ARM New portal (Azure Resource Manager) 

Tools

• Microsoft Azure Storage Explorer
• Azure PowerShell cmdlets

Free EBooks

• Microsoft Azure Essentials: Fundamentals of Azure via Microsoft Press
• Building Cloud Apps with Microsoft Azure via Microsoft Press

Reading to enable the creation of an end to end solution

• Resource Manager Deployment Model
• VPN Gateway how to point to site recovery portal
• Virtual Machines
• Virtual Machines scale sets overview
• Backup configure vault
• Site recovery hyper v site to Azure
• Storage introduction
• What is PaSS
• Traffic Manager Overview
• SQL Database PaSS vs SQL Server
• SQL Database elastic pool
• Cloud Services
• Cloud services how to scale portal
• Active directory what is
• Automation
• Security Centre Introduction
• Log Analytics Overview
• Dev Test Lab
• Virtual Machines Windows Migration Classic Resources Manager

Other Learning Resources: 

• Microsoft Azure Fundamentals via MVA 
• Establish Microsoft Azure IaaS Technical Fundamentals via MVA 
• Dive Deep into Networking Storage and Disaster Recovery Scenarios via MVA 
• Optimize Workload Architecture and Administration Capabilities Within Azure via MVA 
• Azure Friday videos –short videos hosted by Scott Hanselman.  
• Channel9 Microsoft Azure videos –videos on Channel9 
• Microsoft online Azure courses openedx courses