The SparkAI Summit shared a lot of announcements. The open source announcements were
Koalas - a more complete Pandas API
The open sourcing of Databricks Delta as Delta Lake. Delta dramatically simplifies building reliable data lakes on HDFS and cloud storage with ACID transactions, indexes and scalable metadata handling.
Microsoft is joining the MLflow project and adding MLflow APIs in Azure ML.
Rohan Kumar of Microsoft announced .NET for Apache Spark, making Apache Spark accessible to .NET developers - Git Hub
Spark 3.0 expected later in the year
The keynote videos are all online now and other session videos will be there in about 2 weeks.
No comments:
Post a Comment
Note: only a member of this blog may post a comment.