Loading…
This event has ended. View the official site or create your own event → Check it out
This event has ended. Create your own
The attendees list includes all authors (even thought they may not be attending), speakers, artists, etc. 

View the full conference website here:
IEEE Cluster 2013 Conference
View analytic
Wednesday, September 25 • 4:30pm - 4:55pm
Insight and Reduction of MapReduce Stragglers in Heterogeneous Environment

Sign up or log in to save this to your schedule and see who's attending!

Speculative and clone execution are existing techniques to overcome the problems of task stragglers and performance degradation in heterogeneous clusters for big data processing. In this paper, we propose an alternative approach to solving the problems based on analysis results of profiling and the relations of the system parameters. Our approach adjusts the amount of task slots of nodes dynamically to match the processing power of the nodes, according to current task progress rate and resource utilization. It contrasts with the existing techniques by attempting to prevent task stragglers from occurring in the first place through maintaining a balance between resource supply and demand. We have implemented this method in the Hadoop MapReduce platform, and the TPC-H benchmark results show that it achieves 20-30% performance improvement and 35-88% straggler reduction than existing techniques.


Wednesday September 25, 2013 4:30pm - 4:55pm
12th Floor - Circle City 12 (Hilton) 120 W. Market St, Indianapolis, IN

Attendees (3)