MapReduce Based Analysis of Sample Applications Using Hadoop

The rate of increase of structured, semi-structured and unstructured data is very high. To discover hidden information from different types of data is a big challenge. The two techniques, word frequency count and string matching, are applied on a single node and multi node cluster with an input data...

Full description

Saved in:
Bibliographic Details
Published inApplications of Computing and Communication Technologies Vol. 899; pp. 34 - 44
Main Authors Ghazi, Mohd Rehan, Raghava, N. S.
Format Book Chapter
LanguageEnglish
Published Singapore Springer 01.01.2018
Springer Singapore
SeriesCommunications in Computer and Information Science
Subjects
Big data
Cloud computing
Hadoop
HDFS
MapReduce
Online AccessGet full text
ISBN9811320349
9789811320347
ISSN1865-0929
1865-0937
DOI10.1007/978-981-13-2035-4_4

Cover

More Information
Summary:The rate of increase of structured, semi-structured and unstructured data is very high. To discover hidden information from different types of data is a big challenge. The two techniques, word frequency count and string matching, are applied on a single node and multi node cluster with an input data set. The results are analyzed and compared by varying MapReduce configuration of both. In this paper we have tested that for a MapReduce job how changing the number of mappers and reducers can significantly affect performance. Further, it is analyzed how Hadoop invokes number of mappers/reducers depending upon the input size and Hadoop Distributed File System (HDFS) block size. The outcome of research analysis for heterogeneous cluster configurations indicates the prospective of the framework, as well as of mappers and reducers that affect its performance.
ISBN:9811320349
9789811320347
ISSN:1865-0929
1865-0937
DOI:10.1007/978-981-13-2035-4_4