Comments on: Running Hadoop on Windows

By: Brian

Brian — Sun, 24 Jan 2010 20:40:57 +0000

I have followed this,altough trying a pseudo cluster first, and have come to a roadblock when I try to start up hadoop:

$ bin/start-all.sh
starting namenode, logging to /cygdrive/c/apache/hadoop/hadoop-0.20.1/bin/../logs/hadoop-brian-namenode-xx.out

localhost:/bin/bash: /cygdrive/c/apache/hadoop/hadoop-0.20.1/bin/hadoop-daemon.sh: No such file or directory

The error seems related to a carriage return after logging into the same machine()’localhost’ through ssh, running hadoop as a pseudo-cluster

By: Brian

Brian — Mon, 18 Jan 2010 02:45:06 +0000

I’ve started playing with this, as an experiment to try distributing across remote computers, does anyone know if I could use my dreamhost account as a client?

By: Gaurav Kumar

Gaurav Kumar — Sat, 08 Aug 2009 15:39:16 +0000

When I was half way through the article, I thought this must be one of those 10 page guides but to my pleasant surprise you managed to sum it up quite clearly and concisely.
Thank you for the your post.

By: Aseem Puri

Aseem Puri — Fri, 17 Apr 2009 06:08:07 +0000

Can you please tell when configuring SSH to use default usernames what changes have to be made on hadoop-env.sh. I have made a config file and can do ssh. But when i use command bin/start-all.sh it there it still search in differnt directory for hadoop-deamon.sh file

By: Aseem Puri

Aseem Puri — Thu, 02 Apr 2009 07:08:00 +0000

Remove every old data folder where datanode store can store data from your system and the use command bin/hadoop namenode -format

By: Vlad Korolev

Vlad Korolev — Sun, 22 Mar 2009 04:57:33 +0000

I just recently made a tutorial on how to set-up hadoop development environment with Eclipse plug-in on the Windows machine.

http://v-lad.org/Tutorials/Hadoop/00%20-%20Intro.html

By: Avi

Avi — Wed, 11 Feb 2009 20:33:42 +0000

Absolutely great post.
Thanks a lot – you saved me quite a hassle as hadoop’s docs are as good as a map of Africa when you’re stuck in the middle of Alaska…

Cheers!

By: Kim Pham

Kim Pham — Tue, 03 Feb 2009 22:39:07 +0000

If you include package “gzip” when downloading the cygwin packages, you can afterward use the “gunzip” command to extract the hadoop “*.tar.gz” file for “Step 4: Extract Hadoop (All Machines)”.

Run the followings in a cygwin window:
1. Move the “hadoop*.tar.gz’ to /usr/local
2. Run command: gzunzip hadoop*.tar.gz (you will see a hadoop*.tar file)
3. Run command: tar xvf hadoop*.tar

By: Ayon Sinha

Ayon Sinha — Wed, 19 Nov 2008 01:30:51 +0000

One thing I’d like to mention is that while extracting the Hadoop tar file using “smart LF/CR” option in Winzip creates DOS versions of the script which refuses to run on Cygwin bash 3.2. Workarounds (use one):
1. DOnt use the “Smart LF/CR” option in winzip
2. Use tar xvf to extract the files
3. use -o igncr to run bash (painful option)

By: saladin

saladin — Sun, 21 Sep 2008 17:16:30 +0000

Hi Hayes, very good tutorial

You should update it to mention that before running the cluster you MUST format the namenode with the following command

bin/hadoop namenode -format