IBM Research @ Insight 2015 - Insight 2015 - Enabling OpenStack Swift with Spark

Enabling OpenStack Swift with Spark and its Open Source Ecosystem for the Internet of Things

Dalit Naor

OpenStack Swift, which is IBM's object store of choice, provides a simple abstraction to storage with an easy to use RESTful API and is therefore capturing developers. We will present recent work on integrating Swift either as a data source and/or target with a range of open source projects such as Kafka (via Secor), Spark, CouchDB, Tachyon, Hadoop, and ElasticSearch. These connections are available today as open source. Developers who build solutions by combining open source projects with unique business logic can now take advantage of Swift as a data source and a persistent store in composite solutions. The solution is developed by IBM, ATOS & Univ. of Surrey for the EMT Madrid bus company in the context of COSMOS, a European project.