Implementing Hadoop Container Migrations in OpenNebula Private Cloud Environment

• Published in: Role of Edge Analytics in Sustainable Smart City Development pp. 85 - 103
• Main Authors: Kalyanaraman, P, Jothi, K.R, Balakrishnan, P, Navya, R.G, Shah, A, Pandey, V
• Format: Book Chapter
• Language: English
• Published: United States John Wiley & Sons, Incorporated 2020; John Wiley & Sons, Inc
• Subjects: Containers; Docker; Hadoop; Linux; OpenNebula
• ISBN: 9781119681281; 1119681286
• DOI: 10.1002/9781119681328.ch5

The cloud platform provides access to virtual machines, networks, and storage as a service. Virtualization enabled datacenters to pave the way for better resource utilization, server consolidation and scalability. Further, the scalable data‐intensive applications such as Elastic Map Reduce (EMR) are deployed on the cloud. MapReduce uses two operations in programming languages namely, functional map, and reduce. These functions allow us to implement distributed and parallel computing. The methodology used while deploying Hadoop on a virtual cluster is obtained from CloudStack from where virtual machines are obtained, which stresses creating a template based on which all nodes are created. When heterogeneous computing resources are required by the target workloads to satisfy real‐time requirements, virtual Hadoop can be used. The efficiency of Virtual Hadoop is examined and determined. This makes it easier to conduct systematic big data processing by adopting heterogeneous computing. By making use of cloud computing, an efficient and convenient parallel programming environment can be set up to improve resource utilization. Data‐intensive processing can be done in the cloud (virtual machines) using Hadoop, which is an implementation of MapReduce. In a virtual cluster, resource utilization is more efficient when compared to a physical cluster, management is more accessible, power can be saved, and the reliability is improved.