Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Wednesday, 19 November 2025

What is new for Databases and Fabric at Ignite

There is significant news on the database front. Microsoft is unifying its data platform by deeply integrating its databases with Microsoft Fabric, creating a seamless, AI-powered data estate.













Unified Data Estate with Microsoft Fabric

  • Microsoft Fabric now acts as a unified platform that integrates Microsoft’s operational databases (like Azure Cosmos DB, Azure SQL, and PostgreSQL) with analytics and AI tools.
  • This integration enables real-time insights and governed data sharing across operational and analytical systems without complex ETL pipelines.

Real-Time Analytics with Synapse Real-Time Intelligence

  • The new Synapse Real-Time Intelligence in Fabric allows streaming data from operational databases to be analyzed instantly.
  • This supports use cases like fraud detection, predictive maintenance, and personalized recommendations.

Built-in Governance and Security

  • Microsoft Purview is embedded in Fabric, ensuring data lineage, access control, and compliance across the entire data estate.
  • This helps organizations meet regulatory requirements while maintaining agility.

AI-Ready Data for Copilot and Custom Models

  • With this integration, data from operational systems becomes AI-ready by default, enabling seamless use with Microsoft Copilot and custom AI models.
  • This supports natural language querying, automated insights, and generative AI applications.

Fabric IQ 

Fabric IQ is Microsoft Fabric’s new semantic intelligence layer that transforms raw data into business-aware insights by embedding meaning, relationships, and operational context directly into the data platform. This announcement marks the moment Microsoft Fabric evolves from a unified data platform into a unified intelligence platform. It created a shared, real-time understanding of the business. It is designed to solve a core challenge in enterprise AI: AI agents can read data, but they often lack the business context to interpret it meaningfully. Fabric IQ introduces a semantic layer that maps data to real-world business concepts like customers, flights, or inventory, rather than just tables and columns



Fabric IQ introduces a semantic intelligence layer that gives both people and AI the same deep business context your experts rely on. It connects data, meaning, and action through:
  • Ontology - A shared business model built with no-code visual tools
  • Semantic Model - Trusted BI definitions extended into operations & AI
  • Graph Engine - Multi-hop reasoning across connected business systems
  • Data Agent - Virtual analysts that answer questions using real business semantics
  • Operations Agent - Autonomous agents that monitor, reason, and act in real time



Bi-Directional Data Movement

  • Fabric supports bi-directional data movement between databases and the analytics layer, enabling write-back scenarios like updating customer profiles or triggering workflows.

Microsoft Databases and Microsoft Fabric: Your unified and AI-powered data estate

SQL Database in Microsoft Fabric

SQL Database in Microsoft Fabric is a cloud-native, developer-friendly transactional database built on the same engine as Azure SQL Database, designed to integrate seamlessly with the broader Fabric ecosystem.

SQL database in Fabric is built on 3 important pillars: Simple, Autonomous & Secure, and Optimized for AI. It is now genernally available. 

Announcing SQL database in Fabric (Generally Available)







Ignite 2025 Book of News

Microsoft has officially released the Ignite 2025 Book of News. It highlights the capabilities, previews, and security innovations that will empower people to shape the year ahead and enable organizations to change how they work. 


This year’s Microsoft Ignite will focus on exploring the complete lifecycle of AI, creating tools and solutions to drive the next generation of digital transformation as organizations push themselves to unlock creativity and innovation.


Microsoft Ignite 2025 Book of News

SQL Server 2025 is Generally Available

There were a lot of database announcements at Microsoft Ignite which continues to broaden the scope and offerings available. 

The general availability of the next version of SQL Server, SQL Server  2025 was announced. This new version continues to offer best in class security and performance. It is built for AI, made for developers and offer cloud agility through Azure. 

SQL Server 2025 is Now Generally Available

The product just continues to grow.  




The platform architecture






















with these editions






















Other database announcements: 

Azure DocumentDB (GA)
Azure DocumentDB offers AI-ready data, open standards, and multi-cloud deployment. It is a fully managed NoSQL service built on open-source tech and designed for hybrid and multicloud flexibility. It supports advanced search and vector embeddings for more accurate results and is compatible with popular open-source MongoDB drivers and tools.

Azure HorizonDB (Preview)
Azure HorizonDB is a new fully managed PostgreSQL database service.  It runs up to three times faster than open-source PostgreSQL and grows to handle demanding storage requirements with up to 15 replicas running on auto-scaling shared storage.

SQL Database in Fabric (GA)
SQL Database in Fabric bridges the gap between transactional and analytical workloads, enabling scenarios, where data is instantly available for both operational use and analytics, without performance trade-offs or complex ETL pipelines.

Database mirroring in Microsoft Fabric (GA)
The General Availability (GA) of mirroring for Azure SQL Database, Azure SQL Managed Instance, and Cosmos DB. Mirroring for SQL Server 2025 (on-premises or on VMs) was announced as being in Preview. 

Fabric November 2025 Feature Summary | Microsoft Fabric Blog | Microsoft Fabric

Microsoft introduced ecosystem interoperability as a cornerstone of its AI and data strategy. It enables organizations to connect and analyze data across platforms without moving or duplicating it, thus preserving governance, reducing cost, and accelerating insights. It unlocks data estates for AI models to train on live ,governed data, provides business users access insights without waiting for ETL pipelines, as well as reducing complexity for enterprises and cost while improving compliance.



The Key Components Explained
Zero-Copy Integrations
These allow data to be accessed and shared across systems without physically copying it. This is crucial for:
  • Security: Data remains in its original location.
  • Efficiency: No duplication means lower storage and compute costs.
  • Governance: Original data lineage and controls are preserved.
SAP BDC (Business Data Cloud)
Microsoft Fabric now supports bi-directional zero-copy sharing with SAP BDC, giving instant access to semantically rich SAP data for analytics and AI.
Salesforce Data Cloud
Microsoft joins Salesforce’s Zero Copy Partner Network, enabling secure, bidirectional zero-copy integration with platforms like Databricks, Snowflake, and now Microsoft Fabric.
Azure Databricks and Snowflake
Expanded support means these platforms can now participate in zero-copy sharing with Microsoft Fabric, enabling unified analytics across cloud ecosystems.
dbt Jobs in Fabric (Preview)
dbt (data build tool) is now integrated into Fabric, allowing teams to run transformation workflows directly within Microsoft’s unified data platform. This streamlines development and governance for analytics engineers.

Azure at Microsoft Ignite 2025: All the intelligent cloud news explained | Microsoft Azure Blog

Tuesday, 18 November 2025

Microsoft Ignite 2025 Keynote

Microsoft Ignite is in San Francisco, Moscone Center November 18–21, 2025.

The opening keynote featured Judson Althoff, CEO Microsoft commercial business, with senior Microsoft engineering leaders as they unveil the latest innovations driving the next wave of AI transformation that empower every human and every organization across every industry to redefine what it means to become frontier.

In the keynote Judson Althoff talked about frontier transformation and how we need to empower the frontier transformation which is different from AI transformation.

There are 4 common reasons why AI projects fail:

  • Alignment between business and IT professionals is very inconsistent.
  • There are a sea of data quality issues out there that need to be rationalised by AI developers in order to get meaningful output.
  • Governance, requirements and market, whether regulatory or otherwise require AI to be kept on the side-lines from efforts that can impact business.
  • There is an over emphasis on experimentation rather than acts of innovation. AI should be put to work on business areas that can make an impact.

Many businesses who are achieving ROI are using the frontier success framework.




  • Enrich – employee experience
  • Reinvent – customer engagement
  • Reshape – business processes
  • Bend the curve – on innovation

They tie AI back to measurable ROI so the business can see the benefit and they make sure the results are shown in the customer experience journey.

In this new world he mentioned the need to reshape business processes, not simply to apply technology to existing process and expect revolutionary results. It is necessary to examine how business processes can become AI first and change them. Also to focus deeply on innovation and bending the curve to drive competitive advantage. There needs to be a technical focus shift to business focus and holistically reimage the business aligned with human ambitions.

Democratising intelligence through human ambition and AI tools like Copilot and the agent ecosystem  transform how we work, learn, and lead. This AI transformation should do more for humanity.

If you study frontier firms there are 3 traits



  • They put AI in the flow of human ambition: AI is infused in the tools we use
  • Ubiquitous innovation: there is a maker in everyone of us
  • Observability at every layer of the stack: We need to govern and managed the AI that is deployed

Beyond this there are two traits which are fundamental building blocks, Intelligence and Trust. 

Intelligence by putting the 'I' back into AI.  It is how you work, who you work with and how you reason over the data.  It is how we align AI to our purpose to get the outcomes we need. Trust with observability are core.

The keynote set the seen for the event highlighting the human element and that the business processes we have require an over haul to work in new ways. Trust and governance will help set the fundamental building blocks we need to move forward. 

Wednesday, 12 November 2025

Data Toboggan back stories

 It was over 5 years ago we started planning the Data Toboggan event. It has been great to see the event grow 3x and we have changed with the technology growth. We will be opening CfS shortly for the next event anticipated to be 31 January 2025. 


The second main event name was based on a film as it seemed fitting and fun.





From Steam to Silicon to Sentience: Four Industrial Revolutions and the Fragile Future of AI

The story of human progress is punctuated by revolutions, not just in technology, but in how we think, organize, and trust. From the steam engines of the 1840s to the generative models of the 2020s, each wave has promised liberation and delivered disruption. Today, as AI surges toward ubiquity, we must ask: what have we learned from past revolutions, and what must we safeguard before the bubble bursts.



Four Revolutions That Changed Everything

There are four revolutions that resulted in significant change where we can learn from the affects to help the AI revolution progress unhindered.

Era

Catalyst

Impact

Risk

Industrial Revolution (c. 1760 – 1840s)

Steam power, mechanization

Mass production, urbanization, labour displacement

Exploitation, unrest (e.g. Plug Plot Riots, 1842)

Digital Revolution (1950s – 1990s)

Mainframes, UK computing pioneers, PCs

Automation, global communication, software economies

Surveillance, fragmentation, digital exclusion

Cloud Revolution (2000s – 2020s)

Virtualization, SaaS, mobile-first

Scalable infrastructure, remote work, data centralization

Vendor lock-in, opaque governance, cyber risk

AI Revolution (2020s –)

Foundation models, generative AI

Cognitive automation, new interfaces, synthetic creativity

Hallucinations, bias, job loss, trust collapse

 During the industrial revolution there was a deep industrial economic depression. The Plug Plot Riots were a wave of industrial action and disturbances across Lancashire, Cheshire, and Yorkshire, triggered by severe wage reductions (often 20-25% in the cotton and coal industries). Many workers aligned with the Chartist movement advocating for political reform, responded by "plugging" mill boilers, removing drain plugs to flood engines and halt production which forced factories to close.   The Plug Plot Riots of 1842 led to some improvements for workers, notably the prevention of further wage cuts and the eventual passage of the Factory Act 1844, which introduced limited reforms. It introduced a reduction in working hours for women and children, some safety regulations in factories and a modest step toward better labour conditions.

The second revolution of computing was not just technical. It redefined abstraction, logic, and control. From the UK’s early computing pioneers to the rise of PCs, it laid the groundwork for cloud and AI. Yet it also introduced new vulnerabilities: fragmented standards, digital inequality, and the erosion of analogue memory.

Cloud as the Bridge: Infrastructure to Intelligence

Cloud computing connected digital and AI with its abstracted hardware, centralized data, and the capabilities to scale with ease. But as Satya Nadella emphasizes in his annual letter and Microsoft’s 2025 report, innovation without strategic purpose is fragile. Microsoft’s Secure Future Initiative and Quality Excellence Initiative reflect a shift: AI must be built on trust, not just talent.

Brad Smith’s AI Diffusion Report warns that AI is spreading faster than any prior technology but unevenly. The Global South, non-English languages, and underrepresented communities’ risk being left behind.

Data: The Fuel, the Flaw, the Future

AI’s power is unprecedented and has the power to improve or destroy depending on the algorithm development but also on the state of data. Poor quality, biased, or ungoverned data leads to hallucinations, misinformation, and systemic risk. As the BBC’s article on AI hallucinations shows, even the most advanced models can confidently fabricate facts, undermining journalism, science, and public trust. From the simplest things I have seen AI fabricate data, which is written so well, to the untrained eye it could be believed. Once the data is triangulated the output can be trusted. However, the data sources quality, the prompts and data that is behind paywalls will influence the outcome.

This is not a glitch it is a consequence of probabilistic systems trained on imperfect inputs. Without rigorous data governance, provenance tracking, and human oversight, AI becomes a mirror of our worst assumptions.

When the Bubble Bursts: Coping with the AI Comedown

Every revolution has its reckoning. The Plug Plot Riots of 1842, the dot-com crash, and the decline of post-industrial towns all reveal the cost of overhyped promises and underprepared systems. When the AI bubble bursts whether through regulation, disillusionment, or economic correction, organizations with strong data foundations, ethical frameworks, and human-centred design will endure.

Those who chased novelty without governance will falter.

Satya Nadella’s mantra is “thinking in decades, executing in quarters” is more than a business strategy. It’s a survival imperative. The AI era demands long-term vision grounded in short-term accountability. That means:

- Investing in data quality and lineage as core infrastructure

- Embedding responsible AI principles into every product and process

- Preparing workers for augmentation, not just automation

- Designing for resilience, not just scale

Conclusion: From Revolution to Renaissance

The Industrial Revolution reshaped labour. The digital revolution redefined logic. The cloud revolution scaled infrastructure. AI is now rewriting cognition. but without trust, transparency, and governance, even the most powerful tools will falter. As the socio-technical divide deepens and ecological systems strain, the cost of inaction grows, and we risk accelerating collapse socially and ecologically.

The disruption from AI is only just beginning. As Business Insider quoted, “Elon Musk said AI will make desk jobs feel like when workers used to make calculations by hand before the computer age.” This echoes the upheaval of 1842, when industrialisation redefined labour.

If we want AI to be a renaissance, not a reckoning, we must treat data as infrastructure, governance as strategy, and human ethics as non-negotiable. The future isn’t just what we build; it’s what we’re willing to steward.

We must draw a line: to protect data, embed meaningful guardrails, and confront the human cost of displacement. That means planning not only for the jobs we lose, but for the ones we must invent. It also means addressing the widening continental divide in AI development and its cascading impact on the environment and global economy.

References

'It's going to be really bad': Fears over AI bubble bursting grow in Silicon Valley 

https://www.bbc.co.uk/news/articles/cz69qy760weo

Satya Nadella annual letter: Thinking in decades, executing in quarters

https://www.microsoft.com/investor/reports/ar25/index.htmlhttps://www.linkedin.com/pulse/my-annual-letter-thinking-decades-executing-quarters-satya-nadella-7orpc?utm_source=share&utm_medium=member_android&utm_campaign=share_via

Brad Smith https://aka.ms/AIDiffusionReport

Elon Musk says the AI 'supersonic tsunami' will eliminate desk jobs 'at a very rapid pace'

https://www.businessinsider.com/elon-musk-ai-supersonic-tsunami-job-displacement-future-joe-rogan-2025-11

 Transparency: Written with the help of Copilot.

Monday, 3 November 2025

SQLCon 2026

A new conference has arrived. The Microsoft SQL Community Conference, SQLCon 2026, is coming as part of the Microsoft Fabric Community Conference, 16-20 March 2026 in Atlanta. 

It will be a premier level conference for data professionals, featuring 50 breakout sessions and 4 expert-led workshops covering SQL Server, Azure SQL, SQL in Fabric, SQL Tools, migration & modernization, optimization, database security, AI Apps with SQL and much more.

It is a place where the SQL community can come together to share what works, what’s next, and what truly matters.

Register now  https://sqlcon.us/







https://techcommunity.microsoft.com/blog/sqlserver/announcing-sqlcon-2026-better-together-with-fabcon/4466701