3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
Home C Java Hadoop PHP HTML CSS JavaScript jQuery XML JSON Ajax
https://www.javatpoint.com/mapreduce-word-count-example 1/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
MapReduce Word Count Example
In MapReduce word count example, we find out the frequency of each word. Here, the role of Mapper is
to map the keys to the existing values and the role of Reducer is to aggregate the keys of common
values. So, everything is represented in the form of Key-value pair.
Pre-requisite
Java Installation - Check whether the Java is installed or not using the following command.
java -version
Hadoop Installation - Check whether the Hadoop is installed or not using the following
command.
hadoop version
If any of them is not installed in your system, follow the below link to install it.
www.javatpoint.com/hadoop-installation
Steps to execute MapReduce word count example
Create a text file in your local machine and write some text into it.
$ nano data.txt
https://www.javatpoint.com/mapreduce-word-count-example 2/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
Check the text written in the data.txt file.
$ cat data.txt
In this example, we find out the frequency of each word exists in this text file.
https://www.javatpoint.com/mapreduce-word-count-example 3/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
Create a directory in HDFS, where to kept text file.
$ hdfs dfs -mkdir /test
Upload the data.txt file on HDFS in the specific directory.
$ hdfs dfs -put /home/codegyani/data.txt /test
Write the MapReduce program using eclipse.
File: WC_Mapper.java
package com.javatpoint;
import java.io.IOException;
import java.util.StringTokenizer;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
https://www.javatpoint.com/mapreduce-word-count-example 4/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;
public class WC_Mapper extends MapReduceBase implements Mapper<LongWritable,Text,Text,IntWrit
{
private final static IntWritable one = new IntWritable(1);
private Text word = new Text();
public void map(LongWritable key, Text value,OutputCollector<Text,IntWritable> output,
Reporter reporter) throws IOException{
String line = value.toString();
StringTokenizer tokenizer = new StringTokenizer(line);
while (tokenizer.hasMoreTokens()){
word.set(tokenizer.nextToken());
output.collect(word, one);
}
}
File: WC_Reducer.java
package com.javatpoint;
import java.io.IOException;
import java.util.Iterator;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reducer;
import org.apache.hadoop.mapred.Reporter;
public class WC_Reducer extends MapReduceBase implements Reducer<Text,IntWritable,Text,IntWr
public void reduce(Text key, Iterator<IntWritable> values,OutputCollector<Text,IntWritable> outpu
Reporter reporter) throws IOException {
int sum=0;
https://www.javatpoint.com/mapreduce-word-count-example 5/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
while (values.hasNext()) {
sum+=values.next().get();
}
output.collect(key,new IntWritable(sum));
}
}
File: WC_Runner.java
package com.javatpoint;
import java.io.IOException;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.TextInputFormat;
import org.apache.hadoop.mapred.TextOutputFormat;
public class WC_Runner {
public static void main(String[] args) throws IOException{
JobConf conf = new JobConf(WC_Runner.class);
conf.setJobName("WordCount");
conf.setOutputKeyClass(Text.class);
conf.setOutputValueClass(IntWritable.class);
conf.setMapperClass(WC_Mapper.class);
conf.setCombinerClass(WC_Reducer.class);
conf.setReducerClass(WC_Reducer.class);
conf.setInputFormat(TextInputFormat.class);
conf.setOutputFormat(TextOutputFormat.class);
FileInputFormat.setInputPaths(conf,new Path(args[0]));
FileOutputFormat.setOutputPath(conf,new Path(args[1]));
https://www.javatpoint.com/mapreduce-word-count-example 6/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
JobClient.runJob(conf);
}
}
Download the source code.
Create the jar file of this program and name it countworddemo.jar.
Run the jar file
hadoop jar /home/codegyani/wordcountdemo.jar com.javatpoint.WC_Runner /test/data.txt
/r_output
The output is stored in /r_output/part-00000
Now execute the command to see the output.
hdfs dfs -cat /r_output/part-00000
https://www.javatpoint.com/mapreduce-word-count-example 7/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
← Prev Next →
https://www.javatpoint.com/mapreduce-word-count-example 8/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
For Videos Join Our Youtube Channel: Join Now
Feedback
Send your Feedback to feedback@javatpoint.com
Help Others, Please Share
Learn Latest Tutorials
Splunk SPSS Swagger Transact-SQL
Tumblr ReactJS Regex Reinforcement
Learning
R Programming RxJS tutorial React Native Python Design
tutorial tutorial Patterns
RxJS
R Programming React Native Python Design
Patterns
Python Pillow Python Turtle Keras tutorial
tutorial tutorial
Keras
Python Pillow Python Turtle
https://www.javatpoint.com/mapreduce-word-count-example 9/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
Preparation
Aptitude Logical Verbal Ability Interview
Reasoning Questions
Aptitude Verbal Ability
Reasoning Interview Questions
Company
Interview
Questions
Company Questions
Trending Technologies
Artificial AWS Tutorial Selenium Cloud
Intelligence tutorial Computing
AWS
Artificial Selenium Cloud Computing
Intelligence
Hadoop tutorial ReactJS Data Science Angular 7
Tutorial Tutorial Tutorial
Hadoop
ReactJS Data Science Angular 7
Blockchain Git Tutorial Machine DevOps
Tutorial Learning Tutorial Tutorial
Git
Blockchain Machine Learning DevOps
https://www.javatpoint.com/mapreduce-word-count-example 10/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
B.Tech / MCA
DBMS tutorial Data Structures DAA tutorial Operating
tutorial System
DBMS DAA
Data Structures Operating System
Computer Compiler Computer Discrete
Network tutorial Design tutorial Organization and Mathematics
Architecture Tutorial
Computer Network Compiler Design
Computer Discrete
Organization Mathematics
Ethical Hacking Computer Software html tutorial
Graphics Tutorial Engineering
Ethical Hacking Web Technology
Computer Graphics Software
Engineering
Cyber Security Automata C Language C++ tutorial
tutorial Tutorial tutorial
C++
Cyber Security Automata C Programming
Java tutorial .Net Python tutorial List of
Framework Programs
Java Python
tutorial
Programs
.Net
Control Data Mining Data
Systems tutorial Tutorial Warehouse
Tutorial
Control System Data Mining
Data Warehouse
https://www.javatpoint.com/mapreduce-word-count-example 11/12
3/29/24, 4:29 PM MapReduce Word Count Example - javatpoint
https://www.javatpoint.com/mapreduce-word-count-example 12/12
