Welcome

Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein



Tuesday, 20 May 2025

The Shape of Intelligence: Reflections on Microsoft Build 2025

Microsoft Build 2025 took place from 19 to 22 May in Seattle, Washington. It was a four-day event packed with announcements on AI, agentic platforms, and developer tools. 

This year’s Microsoft Build felt less like a tech conference and more like a glimpse into a future already unfolding. What emerged was a tapestry of tools and ideas woven not just for developers, but for thinkers, makers, and leaders who believe technology should feel like an extension of purpose.

Microsoft Fabric continues to evolve and not just as a platform, but as a philosophy. It’s now the connection between data engineering, real-time intelligence, and business insight. The new Digital Twin Builder lets us model the world as we experience it: layered, imperfect, alive. And with Cosmos DB now embedded in Fabric, the boundary between structured and unstructured data feels less rigid, more fluid. It was exciting to see SQL Server grow with another release.

Some of the data and AI announcements from Microsoft Build 2025

AI Agents & Copilot Enhancements

Copilot Tuning – Preview: Customize Copilot with your own data and workflows for domain-specific tasks  

Multi-agent orchestration in Copilot Studio – Preview: Enables agents to collaborate across tasks  

Microsoft Entra Agent ID - Preview: Assigns unique identities to agents for governance  

Agent Store - Generally Available: Discover and deploy agents across Microsoft 365 endpoints  

Bring Your Own Models (BYOM) – Preview: Use over 1,900 Azure AI Foundry models in Copilot Studio

Data Platforms & Analytics

Power BI changes enable you to chat with your data. The preview of Standalone Copilot in Power BI was quietly revolutionary. Now, natural language becomes the interface. It’s not just about data literacy; it’s about data empathy. A new Copilot-powered experience for users to interact with their data through natural language, across any inputs, semantic models, apps and data agents they have access to.

Data agents in Microsoft Fabric serve as virtual business analysts, which you can bring right into Copilot Studio and deploy across Microsoft Teams and Microsoft 365 Copilot

Cosmos DB in Microsoft Fabric – Preview: Adds NoSQL capabilities to Fabric Databases  

Digital Twin Builder in Fabric Real-Time Intelligence – Preview: Create virtual replicas of physical/logical entities  

Standalone Copilot in Power BI – Preview: Natural language interface to query across reports  

Translytical Task Flows in Power BI – Preview: Enables write-back and automation from reports

AI Development & Infrastructure

SQL Server 2025 – Public Preview: AI-ready enterprise database with native integration. It is designed to securely power AI applications—transforming into a vector database in its own right.   

Azure Cosmos DB Vector Search – Preview: Store and query vector embeddings for generative AI  

Azure Database for PostgreSQL with Vector Embeddings – Preview: Generate embeddings via SQL for RAG apps  

Azure AI Foundry Agent Service – Generally Available: Build and orchestrate multi-agent systems  

Model Context Protocol (MCP) – Supported: Open protocol for agentic web interoperability  


To read the complete guide to all the new and events announced this year look at the Book of News

The rise of AI agents was at the heart of Build 2025. Not tools, not assistants but colleagues. With Copilot Tuning, organisations can shape agents using their own language, workflows, and values. It’s low-code, but high-trust. And with multi-agent orchestration, these entities collaborate across domains: HR, IT, marketing.

Open Protocols, Open Possibilities

Microsoft’s embrace of the Model Context Protocol (MCP) and the launch of NLWeb signal a shift toward an open agentic web. It’s a vision where websites speak in semantic layers, and agents navigate with nuance. HTML gave us structure; NLWeb may give us meaning.

Discovery and the Scientific Imagination

Perhaps the most exciting announcement was Microsoft Discovery. An agentic platform for research and development offering a new enterprise agentic platform, which uses specialized agents and graph-based knowledge to accelerate research and development for scientists. 

Build 2025 wasn’t just about what’s new. It was about what’s next and who gets to shape it. 

Tuesday, 8 April 2025

The 2025 AI Index Report

The eighth edition of the AI Index report is out. The 2025 Index is the most comprehensive to date as AI’s influence across society, the economy, and global governance continues to intensify

In the report it covers 

  • in-depth analyses of the evolving landscape of AI hardware 
  • novel estimates of inference costs 
  • new analyses of AI publication and patenting trends. 
  • fresh data on corporate adoption of responsible AI practices
  • expanded coverage of AI’s growing role in science and medicine. 





Monday, 7 April 2025

Why Fabric’s New Health & Quality Capabilities Matter More Than Ever

At FabCon this year, Microsoft doubled down on something many of us in data governance have been saying for a long time: trustworthy data doesn’t happen by accident. It is engineered, monitored, and continuously improved. The newly announced health, quality, and observability capabilities in Microsoft Fabric signal a decisive shift away from reactive firefighting and toward proactive, platform‑level assurance.

For organisations scaling AI, analytics, and operational data products, this matters. Data Quality and Observability are no longer “nice to have”; they are the minimum viable conditions for responsible, repeatable, and compliant data use.

Below is a concise, actionable breakdown of what these new capabilities mean—and how to turn them into immediate value across your estate.

1. Treat Data Health as a First‑Class Operational Signal

Fabric’s expanded health management capabilities give teams something they’ve historically lacked: a unified, platform‑native view of data system health. Instead of stitching together logs, alerts, and manual checks, you now get:

- Integrated telemetry across pipelines, workloads, and storage  

- Early‑warning indicators for degradation, drift, or failure  

- Operational insights that connect system behaviour to business impact  

This elevates data health from a technical afterthought to a governance‑aligned operational metric. For leaders, it means you can finally answer the question: “Is our data estate healthy enough to trust today’s decisions?”

Action: Establish a weekly “Data Health Review” ritual—short, structured, and tied to business outcomes. Treat it like you would a security posture review.

2. Use Data Quality as a Contract, Not a Cleanup Exercise

The new Fabric capabilities reinforce a principle I advocate in every governance programme: quality must be defined, measured, and enforced at the point of creation.

With Fabric’s enhanced quality tooling, teams can now:

- Define expectations (validity, completeness, timeliness) as part of the data product  

- Monitor quality continuously, not periodically  

- Surface issues directly to producers and consumers  

- Build trust signals into downstream AI and analytics workloads  

This shifts quality from reactive cleansing to proactive assurance , a contract between producers and consumers.

Action: Publish a lightweight “Quality Contract” template for all critical data products. Keep it simple: purpose, expectations, checks, and escalation paths.

3. Make Observability the Backbone of AI Governance

As AI workloads scale, observability becomes the difference between responsible innovation and uncontrolled risk. Fabric’s new observability features support:

- Traceability from source to model  

- Lineage‑aware debugging  

- Impact analysis when upstream changes occur  

- Evidence trails for audits, compliance, and Responsible AI reviews  

This is not just operational hygiene, it is AI governance in practice. You cannot assure fairness, accuracy, or safety in AI systems without deep visibility into the data that feeds them.

Action: Integrate Fabric observability outputs into your Responsible AI lifecycle checkpoints—especially model validation and change‑control reviews.

In summary the message from FabCon is clear: health, quality, and observability are now strategic capabilities, not technical chores. For organisations building modern data estates and especially for those embracing AI, where these features are the foundation of trust.









Sunday, 6 April 2025

Microsoft AI Skills Fest

The Microsoft AI Skills Fest is starting from 8 April 2025 to 28 May 2025 to enhance your AI skills and unlock the future. From experiential content to hackathons, there is a lot of opportunity to learn at all levels. 

Register today: https://msft.it/6042qSzcE






Tuesday, 1 April 2025

Microsoft Community Fabric Conference 2025 announcements

There were lots of announcements during the FabCon 2025 conference. Here are a few to catch up on. The main Microsoft blog post can be read here: Fueling tomorrow’s AI with new agentic capabilities and security innovations in Fabric

Fabric Data Factory

Fabric Data Factory enables data ingestion through applying data transformations, and orchestrating different data-related activities for modern data management architectures. There were many new features added such as 

  • General availability for CI/CD capabilities in Data Factory 
  • SPN auth available for pipeline CRUD APIs to secure API apps without needing to use their user token when using the REST API.
  • Lakehouse connector to read delta tables, being able to use the information from the deletion vectors to exclude deleted records.
  • Support for column mapping, and automated auto-creation of tables with new schemas.



Further reading Fabric Data Factory: What’s New and Latest Roadmap

There were also many product improvements and features for Mirroring in Fabric

Mirroring in Fabric – What’s new

Migration Assistant for Fabric Data Warehouse

A really exciting announcement to help migrate from Azure Synapse Analytics was unveiled. This Migration Assistant for Fabric Data Warehouse will soon be in preview! The Migration experience is built natively into Fabric and enables the seamless transition from Azure Synapse Analytics (Data Warehouse) to Microsoft Fabric.

Migration Assistant for Fabric Data Warehouse (Preview)

Copilot for Data Engineering and Data Science

There is a new version of Copilot for Data Engineering and Data Science, a next-generation AI-powered assistant designed to elevate data analysis and coding experience. 

Further reading Enhancing AI productivity in Fabric notebooks with Copilot updates

Copilot F2 and above SKUs 

This is great news for customers on F2 and above SKUs that they will have access to Copilot, Fabric data agents, and more.

hashtag

Azure AI Foundry

Organizations can use Azure AI Foundry to connect customized, conversational agents, created in hashtagMicrosoft Fabric.

Microsoft Purview

There are several new innovations in hashtagMicrosoft Purview for protected, AI-ready data
1. Enhancing Microsoft Purview Data Loss Prevention (DLP) support for lakehouse in Fabric to help prevent sensitive data loss by restricting access
2. Expanding DLP policy support for additional Fabric items such as KQL databases and Mirrored databases to show users notification through policy tips when they are working with sensitive data
3. Microsoft Purview for Copilot in Fabric
4. Data observability, now in preview, within Microsoft Purview Unified Catalog

These are a few of the announcements that were of particular interest to me but there were a lot of announcements showcasing the sheer volume of change and growth of Microsoft Fabric.




Saturday, 29 March 2025

Reflections on MVP Summit

I had a great time at MVP Summit this week.  I learn a lot of new things. It was great to be able to network, provide customer feedback and talk about the ever evolving Data & AI landscape. With the speed of change seemingly increasing all the time it is good to stop and have a clear week reflecting on things that work and don't work and how to improve our practices when working with data and AI.

I think the biggest take away is that the Data & AI community is always approachable and helpful. There is always someone with deep knowledge of an area who can help and the community just gives up so much time to help others learn and grow from helping children learn coding skills, to creating and running conferences, speaking, mentoring and so much more. 

I have been privileged to be a Data Platform MVP for the last 7 years and would like to thank Microsoft for providing an amazing opportunity to attend the conference.




Wednesday, 26 March 2025

Reasoning Agents to help with Research and Analysis

There have been some mind blowing changes released today which will aid the future of work. Microsoft have introduced the first-of-their-kind reasoning agents for work.

Researcher

Researcher helps you tackle complex, multi-step research at work. This will help delivering insights with greater quality and accuracy.



Analyst

Analyst thinks like a skilled data scientist looking at raw data to produce insights in minutes.



Introducing Researcher and Analyst in Microsoft 365 Copilot

Read more about Researcher and Analyst

Deep reasoning and agent flows 

Deep reasoning and agent flows in Microsoft Copilot Studio is in preview and offers a comprehensive platform to create, manage, and deploy agents matched to business needs. 

Deep reasoning

Deep reasoning, agents can perform complex tasks and make more accurate decisions. 

Agent flows

Agent flows bring AI workflows into Copilot Studio that enable agents to follow a predefined sequence of actions, ensuring consistent results for structured tasks.