Heard the latest Hadoop YARN?

YARN is now an Apache sub-project for Hadoop along with HDFS and MapReduce.

Hadoop YARN is a MapReduce upgrade that takes Apache Hadoop beyond MapReduce for data-processing.

Apache Hadoop YARN joins Hadoop Common (core libraries), Hadoop HDFS (storage) and Hadoop MapReduce (the MapReduce implementation) as the sub-projects of the Apache Hadoop which, itself, is a Top Level Project in the Apache Software Foundation.

Arun Murthy (Hortonworks Cofounder) announces the new project in a blog post. Here is his announcement:

As folks are aware, Hadoop HDFS is the data storage layer for Hadoop and MapReduce was the data-processing layer. However, the MapReduce algorithm, by itself, isn’t sufficient for the very wide variety of use-cases we see Hadoop being employed to solve. With YARN, Hadoop now has a generic resource-management and distributed application framework, where by, one can implement multiple data processing applications customized for the task at hand. Hadoop MapReduce is now one such application for YARN and I see several others given my vantage point – in future you will see MPI, graph-processing, simple services etc.; all co-existing with MapReduce applications in a Hadoop YARN cluster.


About Sidney Minassian

Sidney Minassian is the Founder & CEO of Contexti, a premier Big Data Analytics Managed Services company. Prior to Contexti, Sidney founded and led a number of venture capital backed companies in Australia and US that leveraged structured and unstructured data to solve enterprise decision making, collaboration and workflow challenges.
This entry was posted in Big Data & Hadoop, Hadoop, Partner News and tagged , , , . Bookmark the permalink.

Comments are closed.