MapReduce is one of the core framework of Hadoop, it’s computing performance has been widely concerned and researched. In heterogeneous environment, unreasonable map task assignments and inefficient resource utilization lead to multiple backup tasks and the job total execution time is poor.For these problems, this paper proposes a new map task assignment strategy, which is map task dynamic balancing strategy based on file label. The strategy marks on job according to the different types, estimates node computing capabilities and historical processing efficiency of each label task, ensures map task which was assigned can execute successfully. Experiments show that, the strategy can effectively reduce number of backup tasks in map phase, and to some extent optimize the total execution time of the job.
Maximizing Clearance Rate by Penalizing Redundant Task Assignment in Mobile Crowdsensing Auctions
