A picture tells a 1000 words. Just a bit of fun collating some key points about Data Catalogues.
"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein
Tuesday, 1 December 2020
Saturday, 14 November 2020
The day two keynote was delivered by Hanuma Kodavalla a Microsoft Technical Fellow.
"It is still an unending source of surprise for me to see how a few scribbles on a blackboard or on a sheet of paper could change the course of human affairs."
Then sharing the papers that started it all and the fact that is the 50th anniversary of Codd's paper this year.
He mentioned this paper I read a long time ago “One size
fits all": an idea whose time has come and gone M.
He went through all the previous versions of SQL with their key features ending with SQL Server 2019.
Another key paper was mentioned
Then moved to discuss newer product features of SQL Azure Serverless and Azure Defender
Thursday, 12 November 2020
The summit, 10-13 November 2020 is being live streamed through PASSTV.
This year’s first keynote is entitled 'Bringing the future into focus, the end to end Azure Data platform'.
Digital transformation brings change. This change of new technologies can be challenging to learn. For businesses the economies of scale can create efficiencies. The data platform has many elements BI, Analytics and AI, Hybrid Data management, relational databases, Edge and IOT, NoSQL databases, Azure Open Source Database Services (OSS databases) etc. New technologies can help save time. It is good news that the DBA’s key skills are transferable.
Azure services consist of SQL Server on Azure Virtual Machines (for lift and shift and OS level access), Azure SQL Managed Instances (for modernizing existing apps), Azure SQL Database (for build cloud apps). There is also hyperscale for specific use cases. SQL Server workloads run best on Azure and have patching regimes.
It is good to see that Azure helps customers move at their own pace to the cloud which will enable them to pivot their company with less risk to something new. Azure SQL Server serverless means pay only for what is required. Azure Edge is an interesting development and is now generally available.
With the complex network of applications and systems in the modern data landscape it can be difficult to connect the data to gain value. Data virtualization is an important change.
Azure Arc is, I think, a real game changer to help with diverse database locations. It is a set of technologies that extends azure management and native data services outside of azure infrastructure to run across your environment even if you can’t migrate to the cloud due to data sovereignty, latency and or regulatory requirements you can still get the efficiency and agility the cloud offers with Azure Arc.
It is a versionless evergreen SQL that ensures you are always current. It provides cloud elasticity on premises which allows optimization of performance of your workloads and dynamically scale up and down without application downtime. Azure Arc offers unified management which allows you to see your data services running on premises alongside these running on azure through a single pane of glass and manage them using familiar tools like azure portal, azure data studio and azure CLI.
Azure Arc enabled SQL Server Managed Instance and PostgreSQL Hyperscale are in Public Preview
It is possible to use features such as vulnerability assessment and advance threat protection with SQL Defender using the same rules and machine learning algorithms.
The public preview was announced for
- Azure Cosmos DB – Serverless for all APIs
- Azure Database for PostgreSQL – Flexible Server
- Azure Database for MySQL – Flexible Server
- Azure Cache for Redis – Enterprise
Cloud Scale analytics on Azure with Azure Synapse Analytics and Azure Databricks
Announcing public preview of new guided UI for machine learning models. Then to complete the services, Azure Synapse to analyse the data and then report on it in Power BI completes the stack.
Thursday, 1 October 2020
The keynote for the 2020 SQLBits was delivered by Rohan Kumar on 'Digital Transformation from Edge to Cloud with Azure Data' . This celebrated the tools used to achieve digital transformation at breakneck speed within months and shared the innovations just launching. The Azure Data Strategy includes various tools such as cloud databases, Azure Synapse Analytics and Power BI. These tools enable innovation anywhere, on premises, at the edge and in the cloud.
The Azure Data Strategy includes many tools, incorporating SQL Server.
Azure Synapse offers a new class of analytics. There were two announcements Public Preview of Azure Synapse Link for Azure Cosmos DB (Synapse SQL Serverless) and Private preview of Power BI performance accelerator for Azure Synapse Analytics.
Wednesday, 9 September 2020
The UK National Data Strategy has been published https://www.gov.uk/government/publications/uk-national-data-strategy/national-data-strategy There are 4 interconnected pillars listed:
- data foundation
- data skills
- data availability
- responsible data
The National Data Strategy is an ambitious growth for building a world leading data economy. https://www.gov.uk/guidance/national-data-strategy
A summary of the evidence reviewed and evidence gaps can be found here
The mission is to
It builds upon initiatives such as the , the , the and the – setting out a framework for how we approach and invest in data to strengthen our economy and create big opportunities for us in the future.
Sunday, 6 September 2020
It is exciting to see Microsoft Ignite be a digital event experience on September 22-24, 2020. Despite this being a strange year the
The sessions catalog is here.The catalog contains some amazinf learning experiences. The session Building Digital Resilience with Satya Nadella will be an interesting session to watch. I find Satya Nadella such an inspirational speaker.
Have you completed your event list?
🗹 Register 🗹 Download Digital Swag 🗹 Schedule sessions 🗹 Schedule Fun & Wellness breaks 🗹 Get favorite comfy outfit
Thursday, 3 September 2020
Monday, 31 August 2020
To enable data governance programs to be successful it is important to establish the key roles and define the responsibilities within those.
Chief Data Officer - a corporate officer responsible for enterprise-wide governance and utilization of information as an asset, via data processing, analysis, data mining, information trading and other means. Wikipedia
Data Stewards - this label describes accountability and responsibility for data and processes to control the use of data assets. There are varying types of Stewards:
Enterprise data stewards- oversite of the data domain across business functions
Business data stewards - those who are subject matter experts
Technical data stewards - database administrators, BI specialists , data quality administrators
It is necessary the every data set has a data owner. A person responsible for the decision regarding the data. They normal are a business data steward.
Then often there is a data governance steering committee to manage the progress and invoke innovation.
Saturday, 22 August 2020
Data Governance plays a key role in ensuring data is managed. Data Governance as defined by DAMA is
"the exercise of authority and control (planning, monitoring, and enforcement) over the management of data assets. "
The successful management of data requires a program that includes:
Standards and quality
Data management projects for improvement
Data Asset Valuation
The goals of data governance are to bring about a sustainable program of work that is embedded in the day to day management of data. Together this should be measured to ensure improvements can be demonstrated and show positive financial impact.
Azure has Governance features and services to explore.
Wednesday, 12 August 2020
Very excited to here about the change of event from a face to face event to a virtual event. I don't think a year would be complete without SQLBits. With so many virtual tech conferences this year there is so much choice. My number 1 choice is always SQLBits. I have attended and helped at every event since inception. Further information and how to book your tickets: http://sqlbits.com . The event is running 29th September - 3rd October 2020
Sunday, 26 July 2020
The CDEI has published its AI Barometer, a major analysis of the most pressing opportunities, risks, and governance challenges associated with AI and data use in the UK, initially across five sectors (including Criminal Justice, Financial Services, Health & Social Care, Digital & Social Media and Energy & Utilities).
The key findings
Thursday, 23 July 2020
What is a data catalog? Gartner define this as
“A data catalog maintains an inventory of data assets through the discovery, description, and organization of datasets. The catalog provides context to enable data analysts, data scientists, data stewards, and other data consumers to find and understand a relevant dataset for the purpose of extracting business value.”
Data Catalogs are the New Black in Data Management and Analytics (Gartner, 2018)
A data catalog is important to have to record those critical assets that bring value to data. It becomes a library full of core information about your data sources. It can contain a data dictionary and can provide basic statistics about the data. This is a really useful feature being able to explore the data.
- Users can discover the data sources they need and understand the data sources they find. At the same time, Data Catalog help organizations get more value from their existing investments.
- They are inventories of data in the organization
- Data catalogs are a standard for metadata management in the age of big data and advanced analytics
- Adding tags to data sets enable a business glossary of terms to be applied to the data
Thursday, 2 July 2020
Friday, 19 June 2020
Tuesday, 9 June 2020
Wednesday, 3 June 2020
Monday, 1 June 2020
Data Ethics is important to consider and embed in your data system now. With the use of data in AI expanding it is important to obtain create an Ethical Data Handling Strategy.
Monday, 25 May 2020
Saturday, 23 May 2020
The Book of News 2020 shares some amazing advances to help shape the world to come.
The Microsoft Build 2020 Book of News is the guide to the key news items that were announced at Build.
Thursday, 21 May 2020
Wednesday, 20 May 2020
In this time of uncertainty, developers will play a central role in reimaging the world we live in and accelerating our path to recovery. The 3 phases he touched on emergency, recovery and reimaging. Going forward business will need to be able to remote everything at a moments notice, automate everywhere to be agile and simulate anything. The Power Platform, Azure Arc as the first control plane and Teams are enabling the future. Satya left us with a thought provoking statement.
"We are at an inflection point. As developers you have that opportunity, as well as a responsibility, to define what should be rebuilt, what should be reimaged, and what should be left behind." Satya Nadella
We are crossing into a new frontier, anywhere together.
Azure SQL Edge now in preview
Microsoft announces a new supercomputer and lays out vision for future AI work.
It has built one of the top five publicly disclosed supercomputers in the world, making new infrastructure available in Azure to train extremely large artificial intelligence models.