A novel cloud model based data placement strategy for data-intensive application in clouds

• Published in: Computers & electrical engineering Vol. 77; pp. 445 - 456
• Main Authors: Zhang, Xinxin, Hu, Zhigang, Zheng, Meiguang, Li, Jia, yang, Liu
• Format: Journal Article
• Language: English
• Published: Amsterdam Elsevier Ltd 01.07.2019; Elsevier BV
• Subjects: Algorithms; Cloud computing; Cloud model; Computer centers; Computer simulation; Data centers; Data placement; Data transmission; Data-intensive workflow; Mapping; Placement; Servers; Workflow software; Cloud computing; Data-intensive workflow; Cloud model; Data placement
• ISSN: 0045-7906; 1879-0755
• DOI: 10.1016/j.compeleceng.2018.07.007

Today, more and more data-intensive applications are deployed in cloud environments among geo-distributed data centers. It is a fundamental challenge to solve the data placement problem for data-intensive applications. The key problem is to present the uncertain and random process of data placement appropriately, while significantly reducing the transmission time across data centers and decreasing the frequency of data movement among data centers. In this paper, we introduce a new type of entity called a Virtual Data Agent (VDA) and convert the data placement problem into two mapping processes, namely, mapping from the data set to the virtual data agent and mapping from the virtual data agent to the data center. We propose Cloud model based Data Placement Algorithm with Virtual Data Agent (CDPVDA). Through simulation using real workflow applications, we compare CDPVDA with two typical data placement strategies. The results indicate that CDPVDA could reduce unavoidable overhead in data transmission between the data centers by 5% to 20%.