Passionately curious about Data, Databases and Systems Complexity. Data is ubiquitous, the database universe is dichotomous (structured and unstructured), expanding and complex. Find my Database Research at SQLToolkit.co.uk . Microsoft Data Platform MVP

"The important thing is not to stop questioning. Curiosity has its own reason for existing" Einstein

Wednesday, 16 June 2021

Data Toboggan Cool Runnings 21 Summary

The 12 hour event ran on 12 June. We used Teams Live Events. The tool has evolved since we used it in January with some plus points and some behaviour we hadn't seen before. There were some amazing speakers from around the world: Finland, Malta, Australia, Hungary, Serbia, UK (including Scotland), The Netherlands, USA (Seattle, New Mexico), China, (Shanghai)  Canada and Norway. 

It was amazing having Robin Sutara, the Microsoft UK Chief Data Officer present the Keynote. We were also privileged to have a number of product group speakers sharing in depth details about the Azure Synapse features. Then there were community speakers and MVPs speaking. All in all a lot of content with the 45 minutes main sessions wrapped with 5 minute pre recorded lightning talks.  

We had our first expert panel discussion session which I found really interesting and informative. People were able to ask questions in advance using Microsoft forms and during the event using Slido. I would like to thank the people who watched the presentations live on the day and contributed to such a lively chat on Slack. 

I was interested to see where our twitter followers are located. It is interesting to see where Azure Synapse is being used or investigated.

There were many discussions to be had by people on the slack channel. It is nice to have free open discussions on Azure Synapse all day. The links shared in the slack channel were

Presentation tricks : http://blogs.lobsterpot.com.au/2021/01/30/presentation-trickery-online-glassboard-like-lightboard-but-using-just-free-software/ 

Set the utf 8 collation after the dB has been created described by Jovan here https://techcommunity.microsoft.com/t5/azure-synapse-analytics/always-use-utf-8-collations-to-read-utf-8-text-in-serverless-sql/ba-p/1883633

Learn more about distributed execution flow that serverless uses, you can read it in this VLDB article: https://www.vldb.org/pvldb/vol13/p3204-saborit.pdf

Andy C Slides for Turbocharge here https://www.datahai.co.uk/power-bi/turbocharge-power-bi-using-azure-synapse-analytics-session/

Using file metadata in queries : https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/query-specific-files

Best Practices for serverless SQL pool https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/best-practices-serverless-sql-pool

Article that explains the cost management for serverless pools in details: https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/data-processed

Synapse Link for Dataverse: https://docs.microsoft.com/en-us/powerapps/maker/data-platform/export-to-data-lake

Roman Pijack: Cost control for serverless SQL pool - the source code of the views I'll be showing during the lightning talk is available   https://github.com/RomanPijacek/DataToboggan/tree/main/CostControlForServerlessSqlPool

The official MS doc that describes Cost management for serverless SQL pool: https://docs.microsoft.com/en-us/azure/synapse-analytics/sql/data-processed

Craig Porteous: github repo  for his session 


Data Profiler Summary Stats in ADF Data Flows: https://techcommunity.microsoft.com/t5/azure-data-factory/how-to-save-your-data-profiler-summary-stats-in-adf-data-flows/ba-p/1243251

Data Factory to Synapse user voice: https://feedback.azure.com/forums/307516-azure-synapse-analytics/suggestions/41026642-allow-to-share-a-self-hosted-ir-from-data-factory

Well-Architected Framework session slides  https://www.datahai.co.uk/synapse-analytics/applying-the-azure-well-architected-framework-to-azure-synapse-analytics-session/

Mark PM: SQLPackage.exe https://docs.microsoft.com/en-us/sql/tools/sqlpackage/sqlpackage?view=sql-server-ver15 

SqlPackage for Azure Synapse Analytics - SQL Server https://docs.microsoft.com/en-us/sql/tools/sqlpackage/sqlpackage-for-azure-synapse-analytics?view=sql-server-ver15

Wolfgang: First preview this Summer https://devblogs.microsoft.com/visualstudio/visual-studio-2022/

From Drew Skwiers-Koballa  "Build a Data Warehouse with SQL Database Projects": https://github.com/dzsquared/synapse-sqlproj-demo

Finishing with the Azure Synapse Analytics Blog: https://techcommunity.microsoft.com/t5/azure-synapse-analytics/bg-p/AzureSynapseAnalyticsBlog 

No comments:

Post a Comment

Note: only a member of this blog may post a comment.