< Back to BLOG /

Mist 0.5.0 Release – RESTification, HDFS, Simplified DSL, Docker

Share on Facebook4Tweet about this on Twitter0Share on Google+0Share on LinkedIn5
#Mist , #News , #Pulse

We are glad to announce release 0.5.0 of Hydrosphere Mist.

 

Here are the major changes:

 

RESTification

The original idea and motivation behind Hydrosphere Mist is to decouple the data scientist and product/solutions developer by providing a clear API that works for both of them. We are improving that concept by hiding the complex details of Apache Spark job implementation and the location of .jar and Python files into an abstraction called a Router.

 

You could specify a Route as follows:

 

forecast = {
path = ‘hdfs://hdfs-host/jobs/forecast-job.jar’,
className = ‘FastForecast$’,
namespace = ‘production’
}

API users could send a simple REST request and get the results back like this:

POST /api/forecast
{
“param”: 1
}

Read the github documentation for further details.

 

Improved Scala & Python DSL

The improved Mist Job API added syntax sugar (meaning code that is more compact, elegant, and natural) and a unified Scala & Python DSL (Domain Specific Language) to expose the same abstractions. You can read more details of how to wrap your Apache Spark program into a Mist job here.

 

HDFS support for .jar & Python files

Now you can copy your Apache Spark / Hydrosphere Mist job into a HDFS and execute it from there.

 

Improved Documentation & Usability

We have cleaned up the github documentation, added an example SBT project, fixed usability issues with Docker images to make it easier to write code using Mist and test it in a Docker image instead of using Vagrant sandbox

 

Please submit any issues and suggestions at github.