What’s In the Pipeline for Apache Spark?

According to Apache Spark creator Matei Zaharia, Spark will see a number of new features and enhancements to existing features in 2017, including the introduction of a standard binary data format, better integration with Kafka, and even the capability to run Spark on a laptop.

But the big prize—automated creation of continuous applications in Spark—remains a long-term goal. 2016 was a huge year for the Apache Spark project, particularly with the launch of the new Structured Streaming API and ongoing work in Project Tungsten, which started in 2015 to improve performance by escaping the bounds of the JVM.

Author: Alex Woodie


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s