EMR – Training

Simple “Mapper” & “Reducer” Example:

In this example, we are going to use “mapper.py” script to MAP each words on the file “file01.txt” and then use “reducer.py” to count each word.

Step:1 Download/copy the below sample file & Mapper/Reducer python scripts to your local system:

1. Source file to check

https://s3.amazonaws.com/myawsguru/MapReduceExampleScript/file01.txt

https://s3.amazonaws.com/myawsguru/MapReduceExampleScript/mapper.py

https://s3.amazonaws.com/myawsguru/MapReduceExampleScript/reducer.py

Step:2  Add execute permission to mapper.py & reducer.py:

  • chmod 755 mapper.py
  • chmod 755 reducer.py

 

Step:3 Run Mapper script by reading “file01.txt”

  •  cat file01.txt | ./mapper.py

green 1
yellow 1
red 1
blue 1
blue 1
red 1
white 1
white 1
black 1
green 1
red 1
red 1
red 1
Step:3 Run Reducer script

  •  cat file01.txt | ./mapper.py | ./reducer.py

green 1
yellow 1
red 1
blue 2
red 1
white 2
black 1
green 1
red 3

Leave a Comment

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>