What's new for Data Toboggan

There are plenty of things coming from Data Toboggan.

Videos from the last conference are being published on YouTube 

Alpine Coaster call for speakers is open https://sessionize.com/data-toboggan-alpine-coaster-2024/

The Winter conference date to add to your diaries: 25 January 2025.

European Microsoft Fabric Community Conference Keynote

The Microsoft Fabric Community Conference is taking place this week in Stockholm, Sweden. There are 3300 attendees which is amazing.  The Keynote was entitled Microsoft Fabric Vision & Roadmap – Analytics in the Era of AI.  The keynote speakers; Arun Ulag, Kim Manis, Amir Netz, Marco Casalaina, Wangui McKelvey and Rudra Mitra from Microsoft.

The announcements blog contains a vast number of innovative enhancements and capability updates. The blog is: European Fabric Community Conference 2024: Building an AI-powered data platform

Fabric is about providing a unified data platform for AI transformation with these 3 areas

• AI-powered development: Fabric can give teams the AI-powered tools needed for any data project in a pre-integrated and optimized SaaS environment.
• An AI-powered data estate: Fabric can help you access your entire multi-cloud data estate from a single, open data lake, work from the same copy of data across analytics engines, and use that data to power AI innovation 
• AI-powered insights: Fabric can empower everyone to better understand their data with AI-powered visuals and Q&A experiences embedded in the Microsoft 365 apps they use every day. 

Microsoft Fabric is shown in the AI Copilot stack.

There is a new exam available to enhance skilling in data engineering DP-700 This Certification measures Fabric data engineer skills:

• Implementing and managing an analytics solution
• Ingesting and transforming data
• Monitoring and optimizing an analytics solution

Bringing all the skills together for data engineering, data science, and data analytics. You can read more here

 There  are numerous technical announcements

You can see the list of GA features shipped for network security, data security and governance. There is an End-to-End Security Whitepaper to the read aka.ms/FabricSecurityWhitepaper 

The public preview of the new copy job item provides

• Built-in support for copying data between workspaces
• Streamline copying data into and across OneLake
• Setup automatic incremental copy

To help with deployment, the public preview was announced of the Terraform Provider for Microsoft Fabric. This will help empower deployment, management, and governance. This Microsoft Fabric Infrastructure as Code (IaC) new Terraform Provider lets you use the same tools and processes already employed for Microsoft Azure and other cloud resources.

• Check out the provider on the Terraform Registry: https://aka.ms/FabricTF
• To help you get started, there is a handy Quick Starts repository with examples: GitHub Examples for using the Terraform Provider for Microsoft Fabric

Access Azure Databricks Unity Catalog tables directly from Fabric via the new Mirrored Azure Databricks Catalog feature, now in Public Preview https://blog.fabric.microsoft.com/en-US/blog/databricks-unity-catalog-tables-available-in-microsoft-fabric/

A new feature in Fabric: Lakehouse schemas https://blog.fabric.microsoft.com/en-us/blog/organizing-your-tables-with-lakehouse-schemas-and-more-public-preview/

Public Preview of T-SQL Notebook in Fabric  https://blog.fabric.microsoft.com/en-us/blog/announcing-public-preview-of-t-sql-notebook-in-fabric

The General Availability of Copilot for Data Factory in Microsoft Fabric  https://blog.fabric.microsoft.com/en-us/blog/announcing-the-general-availability-of-copilot-for-data-factory-in-microsoft-fabric

All the in dept announcements are on the Fabric Blog. 

The unified data culture was mentioned between Fabric and Office connected through Power BI. 

There are lots more releases out there. This is such an exciting time in the Data and AI space. The exciting announcement is that the conference will be back next year. 

Microsoft Purview at the Fabric Conference

There are a  number of  Microsoft Purview sessions at the European Microsoft Fabric Conference this week in Sweden. 

Check out the schedule 

Good Morning European Microsoft Fabric Community Conference

 Good Morning for day 2 of the European Microsoft Fabric Community Conference

Microsoft Security Community

Microsoft Purview helps in the world against cyber attacks. There is a security community that you can join for updates.  This blog lists some of the useful upcoming webinars in the Microsoft Purview space. 

SEP 18 2024 Microsoft Purview | Microsoft Purview eDiscovery Modern UX

Discussing the new Purview eDiscovery UI, walk through the new features and discuss how the new UI has been optimized and how the new enhanced features that have been added to help simplify and boost user experience.

OCT 15 2024 Microsoft Purview | Navigating the Future of AI: Understanding the EU AI Act and Its Implications

The European Union Artificial Intelligence Act (EU AI Act) is a pioneering regulatory framework designed to oversee the development, deployment, and use of artificial intelligence (AI) technologies across the EU. This landmark legislation aims to ensure AI systems are safe, ethical, and transparent, prioritizing the protection of fundamental rights and user safety.

OCT 16 2024 Microsoft Purview | Introduction to Data Lifecycle Management (DLM) | Compliance Manager | Part 1

Data Lifecycle Management (DLM) is a comprehensive approach to managing data throughout its lifecycle, from creation and storage to archiving and deletion. DLM in Compliance Manager involves a structured approach to managing data through its entire lifecycle within the framework of regulatory and organizational compliance requirements.

Microsoft Purview Data Governance GA Rollout

The new Microsoft  Purview solution is currently being rolled out.  The details on when the region will be available is here: New Microsoft Purview Data Catalog deployment regions (Preview) | Microsoft Learn. 

This new updated tool is critical to make sure data is AI ready across the business. There is a good immersive tour of the Purview Data Governance solution: https://purviewdatagovernance.storylane.io/share/0fu0nmhvphk4

Microsoft Purview: Data Governance Solution is General Availability

The new Microsoft Purview: Data Governance Solution became General Availability 1 September.

This release brings with it a several new capabilities that have added since the public preview launched in April 2024.  This new release focuses on the practice of responsibly federated data governance in built in the application.  The release enables

• modelling and use of durable business concepts to govern evolving and multi-cloud physical data and AI estate
• scale the management of data quality and health in the data estate with the practice of federated data governance, with intelligent automation, and without displacing the people interaction 
• confidently activate responsible discover, understand, and access data in the organization-wide teams and enable users help innovate.