Microsoft Build Fabric: What's new and what's next

The Microsoft Fabric announcements were covered by Amir Netz, Arun Ulagaratchagan, Flavien Daussy, Adam Penhaul. The session is recorded and can be seen here Microsoft Fabric: What's new and what's next.

I live blogged this great main data session at Microsoft Build.

AI is changing the world. AI revolution is based on Data. Data is the fuel that powers AI. It is hard because of the amount of innovation and lots of diversity and complexity.

Purpose built workloads. AI is built into Fabric. Governance is particular important and built in and driven through Microsoft Purview.

Aka.ms/try-fabric

 

There are weekly Fabric released with 60-80 pages of blogs . The roadmap for these features can be found at

Aka.ms/FabricRoadmap

 

What is the point of having data in the lake if no one is using it. It is a bout immediate business access to the data

A SaaS product that looks like Office.  No knobs to optimise Fabric. Results in hours.

• Starts with built-in CI/CD
• Creating deployment pipelines
• And Taskflows (public Preview) to provide help to create things like the medallion architecture.

 

In Fabric you can now bring in partner workloads such as MDM and ESRI. It was announced Microsoft Fabric Workload Development kit as Public Preview.

Al your data, all your teams in one place. You can publish to workload hub for a native fabric workload experience. Aka.ms/FabDevKit

There are multiple methods to get data into Fabric for multi-clouds. Shortcuts to On-Premises Sources for OneLake was announced as Public Preview.

Not everything stored in open formats like databases, so Mirroring helps with this. There is Free Mirroring storage for Replicas. 

Delta format is not the only open format. Iceberg is another major storage function.  There is transparent simultaneous support of Delta Lake and Iceberg formats just announced. It is now possible to also connect to Salesforce and not move the data.  Also now an expanded partnership with snowflake and Adobe.

To have unified API with the public preview of the developer friendly API for GraphQL to all data in OneLake. (GraphQL uses JSON structures).

Unified data culture requires real time data. Microsoft announced Real-Time Intelligence. It uses the the Real Time hub powered by AI for data in motion. (OneLake data hub is for date at Rest)

So Real-Time Intelligence in the real world.

Copilot is integrated in every Microsoft Fabric Experience. Copilot in Fabric is now Generally Available.  This means AI driven insights drive insights out of the box and with custom generative AI for your data. 

Announcing Public Preview of AI Skills in Fabric.  It allows you to build your own Generative AI in Fabric

Simple to get started

• Create AI Skill
• Add data – ground in data
• Select tables to ground the data

Query in natural language

In conclusion come and Join the Microsoft Fabric Team in Stockholm, Sweden 24-27 September 2024

Aka.ms/FabCon-Europe

Responsible AI Transparency Report

Microsoft have shared how they work with AI responsible in this paper  Responsible AI Transparency Report How we build, support our customers, and grow.  The report outlines Microsoft’s approach to building generative AI applications responsibly, adhering to six core values of transparency, accountability, fairness, inclusiveness, reliability and safety, and privacy and security.  The framework is all based around the govern, map, measure and manage cycle.  

Govern 

Establishes the context for AI risk management, including adherence to policies and pre-deployment reviews.

• Policies and principles
• Procedures for pre-trained models
• Stakeholder coordination
• Documentation
• Pre-deployment reviews

Map 

Involves identifying and prioritizing AI risks and conducting impact assessments to inform decisions.

• Responsible AI Impact Assessments
• Privacy and security review
• Red teaming

Measure

Implements procedures to assess AI risks and the effectiveness of mitigations through established metrics.

• Metrics for identified risks
• Mitigations performance testing

Manage

Focuses on mitigating identified risks at both the platform and application levels, with ongoing monitoring and user feedback.

• User agency
• Transparency
• Human review and oversight
• Managing content risks
• Ongoing monitoring
• Defense in depth

These are all depicted in the diagram in the paper which is a very informative read.

References

Responsible AI Transparency Report How we build, support our customers, and grow

https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RW1l5BO

Responsible AI – A Data Governance Approach

I am speaking at the Bath Azure User Group meeting about Responsible AI - a Data Governance approach. I see Responsible AI a subset of Data Governance. This session covers where we are with legislation and tools, why good data quality is a must for AI and how to get started. 

Data Governance and Responsible AI, and the embellishment of AI within Microsoft Purview aid and prepare business for using AI. Moving forward I believe that combining the use of both Data Governance and Responsible AI into one actionable framework that  it will bring immediate rewards to every business use case.

Hope you can join us join us 22 May 2024 18-20 in Bath

https://lnkd.in/eRT8RijE