How to Remote Debug Standalone Hadoop

  When you run your map reduce applications you may have hiccups here and there and may need to have a look inside. If you need to remote debug rather going through logs and figuring out what went wrong following is the procedure.

I am using Intellij Idea as the IDE but for other IDE's process is similar.

1) In Intellij Idea go to Run > Edit Configuration and then click on "+" . And then add Remote for "Remote Debugging"

2) You will have following window after clicking on Remote. You can change the port you are using for remote debugging in this panel.

3) Open your Hadoop Root folder and navigate to etc/hadoop-env.sh in your editor. At the bottom of the file add the following line. (Make sure to have the port you given for IDE configuration as the address)

export HADOOP_OPTS="-agentlib:jdwp=transport=dt_socket,server=y,suspend=y,address=5005"

Now you can start hadoop in standalone mode and it will wait until you attach your IDE to debug process to resume.