Hadoop版本0.20.2
Hbase版本0.90.2
集群有3個節點 hdp0~hdp2
hdp0為namenode 及Hmaster
啟動完Hadoop,Hbase後
用jps查詢hdp0為
代碼:
hadoop@hdp0:~$ jps
4571 JobTracker
4727 TaskTracker
6862 Jps
6015 HQuorumPeer
4497 SecondaryNameNode
4169 NameNode
4337 DataNode
測試HDFS可用put指令上傳檔案
以下是Hmaster的log
代碼:
二 9月 10 13:26:18 CST 2013 Starting master on hdp0
ulimit -n 1024
2013-09-10 13:26:19,306 INFO org.apache.hadoop.hbase.ipc.HBaseRpcMetrics: Initializing RPC Metrics with hostName=HMaster, port=60000
2013-09-10 13:26:19,375 INFO org.apache.hadoop.hbase.security.User: Skipping login, not running on secure Hadoop
2013-09-10 13:26:19,376 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server Responder: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server listener on 60000: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000: starting
2013-09-10 13:26:19,378 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000: starting
2013-09-10 13:26:19,389 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000: starting
2013-09-10 13:26:19,390 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000: starting
2013-09-10 13:26:19,394 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000: starting
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.3.2-1031432, built on 11/05/2010 05:32 GMT
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=hdp0
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.6.0_24
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc.
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-6-openjdk/jre
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/home/hadoop/hbase-0.90.2/bin/../conf:/usr/lib/jvm/java-6-openjdk/lib/tools.jar:/home/hadoop/hbase-0.90.2/bin/..:/home/hadoop/hbase-0.90.2/bin/../hbase-0.90.2.jar:/home/hadoop/hbase-0.90.2/bin/../hbase-0.90.2-tests.jar:/home/hadoop/hbase-0.90.2/bin/../lib/activation-1.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/asm-3.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/avro-1.3.3.jar:/home/hadoop/hbase-0.90.2/bin/../lib/commons-cli-1.2.jar:/home/hadoop/hbase-0.90.2/bin/../lib/commons-codec-1.4.jar:/home/hadoop/hbase-0.90.2/bin/../lib/commons-el-1.0.jar:/home/hadoop/hbase-0.90.2/bin/../lib/commons-httpclient-3.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/commons-lang-2.5.jar:/home/hadoop/hbase-0.90.2/bin/../lib/commons-logging-1.1.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/commons-net-1.4.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/core-3.1.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/guava-r06.jar:/home/hadoop/hbase-0.90.2/bin/../lib/hadoop-0.20.2-core.jar:/home/hadoop/hbase-0.90.2/bin/../lib/hbase-0.90.2.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jackson-core-asl-1.5.5.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jackson-jaxrs-1.5.5.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jackson-mapper-asl-1.4.2.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jackson-xc-1.5.5.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jasper-compiler-5.5.23.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jasper-runtime-5.5.23.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jaxb-api-2.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jaxb-impl-2.1.12.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jersey-core-1.4.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jersey-json-1.4.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jersey-server-1.4.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jettison-1.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jetty-6.1.26.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jetty-util-6.1.26.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jruby-complete-1.6.0.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jsp-2.1-6.1.14.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jsp-api-2.1-6.1.14.jar:/home/hadoop/hbase-0.90.2/bin/../lib/jsr311-api-1.1.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/log4j-1.2.16.jar:/home/hadoop/hbase-0.90.2/bin/../lib/protobuf-java-2.3.0.jar:/home/hadoop/hbase-0.90.2/bin/../lib/servlet-api-2.5-6.1.14.jar:/home/hadoop/hbase-0.90.2/bin/../lib/slf4j-api-1.5.8.jar:/home/hadoop/hbase-0.90.2/bin/../lib/slf4j-log4j12-1.5.8.jar:/home/hadoop/hbase-0.90.2/bin/../lib/stax-api-1.0.1.jar:/home/hadoop/hbase-0.90.2/bin/../lib/thrift-0.2.0.jar:/home/hadoop/hbase-0.90.2/bin/../lib/xmlenc-0.52.jar:/home/hadoop/hbase-0.90.2/bin/../lib/zookeeper-3.3.2.jar
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/jvm/java-6-openjdk/jre/lib/i386/client:/usr/lib/jvm/java-6-openjdk/jre/lib/i386:/usr/lib/jvm/java-6-openjdk/jre/../lib/i386:/usr/java/packages/lib/i386:/usr/lib/jni:/lib:/usr/lib
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=i386
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=2.6.32-33-generic
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=hadoop
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop
2013-09-10 13:26:19,404 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/home/hadoop/hbase-0.90.2
2013-09-10 13:26:19,405 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=hdp2:2222,hdp1:2222,hdp0:2222 sessionTimeout=180000 watcher=master:60000
2013-09-10 13:26:19,413 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server hdp1/192.168.0.101:2222
2013-09-10 13:26:19,421 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to hdp1/192.168.0.101:2222, initiating session
2013-09-10 13:26:19,428 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server hdp1/192.168.0.101:2222, sessionid = 0x141064ec7920001, negotiated timeout = 180000
2013-09-10 13:26:19,452 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=Master, sessionId=hdp0:60000
2013-09-10 13:26:19,458 INFO org.apache.hadoop.hbase.metrics: MetricsString added: revision
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUser
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsDate
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsUrl
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: date
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsRevision
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: user
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: hdfsVersion
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: url
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: MetricsString added: version
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.metrics: new MBeanInfo
2013-09-10 13:26:19,459 INFO org.apache.hadoop.hbase.master.metrics.MasterMetrics: Initialized
2013-09-10 13:26:19,465 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Master=hdp0:60000
2013-09-10 13:26:19,666 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: hdfs://hdp0:9000.oldlogs
at org.apache.hadoop.fs.Path.initialize(Path.java:140)
at org.apache.hadoop.fs.Path.<init>(Path.java:71)
at org.apache.hadoop.fs.Path.<init>(Path.java:50)
at org.apache.hadoop.hbase.master.MasterFileSystem.<init>(MasterFileSystem.java:90)
at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:347)
at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:283)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: hdfs://hdp0:9000.oldlogs
at java.net.URI.checkPath(URI.java:1804)
at java.net.URI.<init>(URI.java:752)
at org.apache.hadoop.fs.Path.initialize(Path.java:137)
... 5 more
2013-09-10 13:26:19,667 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2013-09-10 13:26:19,668 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping service threads
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 60000
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000: exiting
2013-09-10 13:26:19,668 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000: exiting
2013-09-10 13:26:19,669 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 60000
2013-09-10 13:26:19,670 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder
2013-09-10 13:26:19,756 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down
2013-09-10 13:26:19,757 INFO org.apache.zookeeper.ZooKeeper: Session: 0x141064ec7920001 closed
2013-09-10 13:26:19,757 INFO org.apache.hadoop.hbase.master.HMaster: HMaster main thread exiting
Hbase config檔內容如下
代碼:
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://hdp0:9000</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2222</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>hdp0,hdp1,hdp2</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/home/hadoop/hbase-data</value>
</property>
<property>
<name>hbase.tmp.dir</name>
<value>/home/hadoop/hbase-${user-name}</value>
</property>
<property>
<name>hbase.master</name>
<value>hdp0:60000</value>
</property>
</configuration>
想請教各位不知道是否哪有設定錯誤
懇請指點