February 07, 2014

Hadoop Mapper only MapReduce jobs

It's funny! You can have mapper only Hadoop MapReduce Jobs. This would be useful sometime when you need to change the structure of data. Otherwise you can use this way when want to filter out data. But I don't think you will need this much often.

First file is a simple mapper which really does nothing. You can change it as you want. You can see in main method job.setNumReduceTasks(0); line which set reduce tasks to 0. You can find Maven project on GitHub

4 comments:

  1. It does not work. Many errors
    Caused by: java.lang.NoSuchMethodException: App$Map.()

    ReplyDelete
    Replies
    1. You need to write your map class. I think you don't have any map class.

      Delete
  2. java.lang.RuntimeException: java.lang.NoSuchMethodException: App$Map.()
    Caused by: java.lang.NoSuchMethodException: App$Map.()

    ReplyDelete