Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
The attendees list includes all authors (even thought they may not be attending), speakers, artists, etc. 

View the full conference website here:
IEEE Cluster 2013 Conference
View analytic
Wednesday, September 25 • 4:00pm - 4:25pm
Dynamic Slot Allocation Technique for MapReduce Clusters (unable to attend, Video/Skype Presentation)

Sign up or log in to save this to your schedule and see who's attending!

MapReduce is a popular parallel computing paradigm for large-scale data processing in clusters and data centers. However, the slot utilization can be low, especially when Hadoop Fair Scheduler is used, due to the pre-allocation of slots among map and reduce tasks, and the order that map tasks followed by reduce tasks in a typical MapReduce environment. To address this problem, we propose to allow slots to be dynamically (re)allocated to either map or reduce tasks depending on their actual requirement. Specifically, we have proposed two types of Dynamic Hadoop Fair Scheduler (DHFS), for two different levels of fairness (i.e., cluster and pool level. The experimental results show that the proposed DHFS can improve the system performance significantly (by 32%~55% for a single job and 44%~68% for multiple jobs) while guaranteeing the fairness.


Wednesday September 25, 2013 4:00pm - 4:25pm
12th Floor - Circle City 12 (Hilton) 120 W. Market St, Indianapolis, IN

Attendees (2)