IBM Research @ Insight 2015 - Insight 2015 - Accelerating Machine Learning

Accelerating Machine Learning applications on Spark using GPUs

Wei Tan

Apache Spark is best known for high speed in-memory processing of big data applications. In this talk, we focus on machine learning applications and present an IBM Research project that demonstrates significantly enhanced Spark performance by exploiting attached GPU accelerators. We will illustrate the performance gains that can be achieved by accelerating core Spark operations using GPUs in a fully transparent manner, that is, without requiring any modifications of user application code. Finally, we will discuss our Spark acceleration roadmap to achieve continued performance improvements by exploiting multiple GPUs on each node, a high speed IBM Power Systems CPU and the Nvidia GPU link called NVLINK.