The keynotes for the European Spark and AI summit 2019 which took place 16 October 2019 in Amsterdam are online to watch if you missed the event. A few of the topics are
Unified Data Analytics: Helping Data Teams Solve the World’s Toughest Problems
New Developments in the Open Source Ecosystem
Saving Energy in Homes with a Unified Approach to Data and AI
Simplifying Model Management with MLflow
Scalable AI for Good
Forecasting 'What-if' Scenarios in Retail Using ML-Powered Interactive Tools
Democratizing Machine Learning: Perspective from a scikit-learn Creator
Imaging the Unseen: Taking the First Picture of a Black Hole
Databricks Simplifies Machine Learning Model Management At Scale With MLflow Model Registry
Databricks brings its Delta Lake project to the Linux Foundation
A design pattern is emerging called 'Lake House'. This is a pattern where Spark is not just replacing Hadoop and ETL, but also associated with data warehouses, business intelligence and reporting.
Delta Lake is an open source storage layer that brings reliability to data lakes. Delta Lake provides ACID transactions, scalable metadata handling, and unifies streaming and batch data processing. Delta Lake runs on top of your existing data lake and is fully compatible with Apache Spark APIs.