Taiwan Hadoop Forum

想請問Jazz大，及有用 Virtualbox 架設過叢集的大哥們
我在用Virtualbox架設三台叢集時，我的Node一直起不來。設定應該確定沒錯了，但是一開始，假如是馬上 ./stop-all.sh至少還可以看得出來tasktracker都有起來，但是datanode都沒起來。再隔一段時間測試時，發現 ./start-all.sh後等待一段時間再 ./stop-all.sh，不但tasktracker都沒起來，是連datanode都根本沒有起來!! 就連我在node1上看50030也是，Nodes數都只有1。
同時也發現，假如一開始在node1 ./start-all.sh跑完後，馬上觀察node2的jps(指令)會有TaskTracker，但是node3都不會出現TaskTracker進程!!
(如下有截取片段測試結果)及(各個該設定的設定檔設定)
所以，想請教幾個問題：

1. 從 ./start-all.sh 的執行結果後的角度上，應該是設定沒錯，那為什麼節點會起不來?
2. 如前言所提及，不但tasktracker都沒起來，是連datanode都根本沒有起來!! 想請問為什麼datanode在不論重複多少次測試都一樣起不來?
3. 是不是用Virtualbox架設叢集，『難免』或是『一定會』碰到『Node起不來』的問題?
4. 有沒有什麼好建議假如要練習自行搭建叢集，但node不怕起不來的方式? 或是有什麼可練習自行搭建叢集的軟體? 我爬過文章後，多數都是Virtualbox測試居多的說!?
5. 請問在Virtualbox架設叢集的操作設定檔設定，跟在一般真正實體電腦的設定檔設定，是一樣的嗎?

# vim /opt/hadoop/conf/hadoop-env.sh
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_25
export HADOOP_HOME=/opt/hadoop
export HADOOP_CONF_DIR=/opt/hadoop/conf

# vim /opt/hadoop/conf/core-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://node1:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/var/hadoop/hadoop-${user.name}</value>
</property>
</configuration>

# vim /opt/hadoop/conf/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>

# vi m /opt/hadoop/conf/mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>node1:9001</value>
</property>
</configuration>

# vim /opt/hadoop/conf/masters
node1

# vim /opt/hadoop/conf/slaves
node1
node2
node3


[root@node1 bin]# ./start-all.sh
starting namenode, logging to /opt/hadoop/logs/hadoop-root-namenode-node1.out
node3: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-node3.out
node2: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-node2.out
node1: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-node1.out
node1: starting secondarynamenode, logging to /opt/hadoop/logs/hadoop-root-secondarynamenode-node1.out
starting jobtracker, logging to /opt/hadoop/logs/hadoop-root-jobtracker-node1.out
node3: starting tasktracker, logging to /opt/hadoop/logs/hadoop-root-tasktracker-node3.out
node2: starting tasktracker, logging to /opt/hadoop/logs/hadoop-root-tasktracker-node2.out
node1: starting tasktracker, logging to /opt/hadoop/logs/hadoop-root-tasktracker-node1.out
[root@node1 bin]#
[root@node1 bin]# ./stop-all.sh
stopping jobtracker
node3: stopping tasktracker
node2: stopping tasktracker
node1: stopping tasktracker
stopping namenode
node3: no datanode to stop
node2: no datanode to stop
node1: stopping datanode
node1: stopping secondarynamenode

第二次測試(隔一段時間後)
[root@node1 bin]# ./start-all.sh
starting namenode, logging to /opt/hadoop/logs/hadoop-root-namenode-node1.out
node2: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-node2.out
node3: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-node3.out
node1: starting datanode, logging to /opt/hadoop/logs/hadoop-root-datanode-node1.out
node1: starting secondarynamenode, logging to /opt/hadoop/logs/hadoop-root-secondarynamenode-node1.out
starting jobtracker, logging to /opt/hadoop/logs/hadoop-root-jobtracker-node1.out
node3: starting tasktracker, logging to /opt/hadoop/logs/hadoop-root-tasktracker-node3.out
node2: starting tasktracker, logging to /opt/hadoop/logs/hadoop-root-tasktracker-node2.out
node1: starting tasktracker, logging to /opt/hadoop/logs/hadoop-root-tasktracker-node1.out
[root@node1 bin]#
[root@node1 bin]# ./stop-all.sh
stopping jobtracker
node3: no tasktracker to stop
node2: no tasktracker to stop
node1: stopping tasktracker
stopping namenode
node3: no datanode to stop
node2: no datanode to stop
node1: stopping datanode
node1: stopping secondarynamenode

感恩

hadoop-root-datanode-node2.log 和 hadoop-root-datanode-node3.log 的 Error 一樣
2013-11-07 01:01:37,716 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG: host = node3/127.0.0.1
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.20.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branche
s/branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
2013-11-07 01:01:41,757 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 0 time(s).
2013-11-07 01:01:42,760 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 1 time(s).
2013-11-07 01:01:43,763 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 2 time(s).
2013-11-07 01:01:44,765 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 3 time(s).
2013-11-07 01:01:45,767 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 4 time(s).
2013-11-07 01:01:49,769 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 5 time(s).
2013-11-07 01:01:50,771 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 6 time(s).
2013-11-07 01:01:51,773 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 7 time(s).
2013-11-07 01:01:52,777 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 8 time(s).
2013-11-07 01:01:53,781 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: node1/192.168.10.1:9000. Already tried 9 time(s).
2013-11-07 01:01:53,786 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
Call to node1/192.168.10.1:9000 failed on local exception: java.net.NoRouteToHostException: No route to host
at org.apache.hadoop.ipc.Client.wrapException(Client.java:775)
at org.apache.hadoop.ipc.Client.call(Client.java:743)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
at com.sun.proxy.$Proxy4.getProtocolVersion(Unknown Source)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:346)
at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:383)
at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:314)
at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:291)
at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:269)
at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:216)
at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1283)
at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)
Caused by: java.net.NoRouteToHostException: No route to host
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:708)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304)
at org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:860)
at org.apache.hadoop.ipc.Client.call(Client.java:720)
... 13 more
2013-11-07 01:01:53,789 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at node3/127.0.0.1
************************************************************/


hadoop-root-tasktracker-node2.log 和 hadoop-root-tasktracker-node3.log 的 Error 也一樣
2013-11-07 01:01:37,643 INFO org.apache.hadoop.mapred.TaskTracker: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG: host = node3/127.0.0.1
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.20.2
STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop ... /branches/
branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
2013-11-07 01:01:39,697 ERROR org.apache.hadoop.mapred.TaskTracker:
Can not start task tracker because java.lang.RuntimeException: Not a host:port pair: local
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:136)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:123)
at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:1807)
at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:905)
at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2833)
2013-11-07 01:01:39,699 INFO org.apache.hadoop.mapred.TaskTracker: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at node3/127.0.0.1
************************************************************/

不知道能不能告訴小的No route to host和 Not a host:port pair: local...各是在講什麼錯誤的說?

> 1. 從 ./start-all.sh 的執行結果後的角度上，
> 應該是設定沒錯，那為什麼節點會起不來?

從討論看起來，SSH 會通，但節點起不來。而 Log 顯示是網路問題(No route...)。首先得確定您的網路配置方式是 Bridge, NAT 還是 Host-Only。您是否有自行做 22 port 的 port forward ？

> 2. 如前言所提及，不但tasktracker都沒起來，
> 是連datanode都根本沒有起來!!
> 想請問為什麼datanode在不論重複多少次測試都一樣起不來?

先把網路問題確認後再說吧。

> 3. 是不是用Virtualbox架設叢集，『難免』或是『一定會』碰到『Node起不來』的問題?

問題不能這樣推論，並不是工具不好，而是「易用性」確實還沒有那麼好。
不管用 VirtualBox 或 VMWare, Hyper-V 我想首先開發者必須有基本的網路觀念，因為 Bridge, NAT, Host-Only 三種模式常讓初學者搞不清楚狀況。簡單來說 Bridge 就是您的 VM 會跟實體主機接在同一個實體網路孔上，若三台都是 Bridge，那就得要有路由(router)有 Gateway 幫忙轉才行。若是 NAT，就視為三台 VM 是接在家用 IP 分享器。若是 Host-Only 就像接在一台有 DHCP 的 Switch 上，出不去。

> 4. 有沒有什麼好建議假如要練習自行搭建叢集，
> 但node不怕起不來的方式? 或是有什麼可練習自行
> 搭建叢集的軟體? 我爬過文章後，
> 多數都是Virtualbox測試居多的說!?

還是回歸到叢集環境本來所需的背景知識就比較多，這也是我們在推動社群時，一直希望以降低這個門檻為首務。

> 5. 請問在Virtualbox架設叢集的操作設定檔設定，
> 跟在一般真正實體電腦的設定檔設定，是一樣的嗎?

一樣的。

- Jazz

你虚拟机的网卡MAC地址是不是冲突，网络配置不对