Adaptive Task Scheduling Strategy Based on Dynamic Workload Adjustment for Heterogeneous Hadoop Clusters

Xiaolong Xu, Lingling Cao, Xinheng Wang

Research output: Contribution to journalArticlepeer-review

34 Citations (Scopus)


The original task scheduling algorithm of Hadoop cannot meet the performance requirements of heterogeneous clusters. According to the dynamic change of load of each task node and the difference of node performance of different tasks in the heterogeneous Hadoop cluster, a novel adaptive task scheduling strategy based on dynamic workload adjustment (ATSDWA) is presented. With ATSDWA, tasktrackers can adapt to the change of load at runtime, obtain tasks in accordance with the computing ability of their own, and realize the self-regulation, while avoiding the complexity of algorithm, which is the prime reason to make jobtracker the system performance bottleneck. Experimental results show that ATSDWA is a highly efficient and reliable algorithm, which can make heterogeneous Hadoop clusters stable, scalable, efficient, and load balancing. Furthermore, its performance is superior to the original and improved task scheduling strategy of Hadoop, from the aspects of the execution time of tasks, the resource utilization, and the speed-up ratio.

Original languageEnglish
Article number6832443
Pages (from-to)471-482
Number of pages12
JournalIEEE Systems Journal
Issue number2
Publication statusPublished - Jun 2016
Externally publishedYes


  • Adaptive scheduling
  • clustering methods
  • computational efficiency
  • distributed computing
  • dynamic scheduling


Dive into the research topics of 'Adaptive Task Scheduling Strategy Based on Dynamic Workload Adjustment for Heterogeneous Hadoop Clusters'. Together they form a unique fingerprint.

Cite this