Hadoopy 0.4.0

Operating systemsOS : Windows / Linux / Mac OS / BSD / Solaris
Program licensingScript Licensing : GPL - GNU General Public License
CreatedCreated : Jul 13, 2011
Size downloadDownloads : 1
Program licensing
Thank you for voting...

It is basically a Python library for MapReduce ...

It is basically a python library for MapReduce written in Cython.
Most important functions of Hadoopy by Brandyn White:

• Similar interface as the Hadoop API (design patterns usable between Python/Java interfaces)

• General compatibility with dumbo to allow users to switch back and forth

• Usable on Hadoop clusters without Python or admin access

• Fast conversion and processing

• Stay small and well documented

• Be transparent with what is going on

• Handle programs with complicated . so’s, ctypes, and extensions

• Code written for hack-ability

• Simple hdfs access (e. g. , reading, writing, ls)

• Support (and not replicate) the greater Hadoop ecosystem (e. g. , oozie, whirr)

• Automated job parallelization ‘auto-oozie’ available in the hadoopy 0.4.0 flow project (maintained out of branch)

• Local execution of unmodified MapReduce job with launch_local

• Read/write sequence files of TypedBytes directly to HDFS from python (readtb, writetb)

• Allows printing to stdout and stderr in Hadoop tasks without causing problems (uses the ‘pipe hopping’ technique, both are available in the task’s stderr)

• Works on clusters without any extra installation, Python, or any Python libraries (uses Pyinstaller that is included in this source tree)

• Works on OS X

• Critical path is in Cython

• Simple HDFS access (readtb and ls) inside Python, even inside running jobs

• Unit test interface

• Reporting using status and counters (and print statements! no need to be scared of them in Hadoopy [hadoopy0.4.0.exe])

• Supports design patterns in the Lin&Dyer book

• Typedbytes support (very fast)

• Oozie support
Demands:

• Cython 0/13 or higher

Hadoopy 0.4.0 scripting tags: mapreduce library, clusters, hadoopy, database wrapper, hdfs, access, python, apache hadoop, oozie. What is new in Hadoopy 0.4.0 software script? - Unable to find Hadoopy 0.4.0 news. What is improvements are expecting? Newly-made Hadoopy 0.5 will be downloaded from here. You may download directly. Please write the reviews of the Hadoopy. License limitations are unspecified.