Taiwan Hadoop Forum
http://forum.hadoop.tw/

HBase replication
http://forum.hadoop.tw/viewtopic.php?f=4&t=38064
1 頁 (共 1 頁)

發表人:  nowitzki0205 [ 2013-11-10, 13:13 ]
文章主題 :  HBase replication

想請問一下HBase關於Replication 配置環境問題

1. zookeeper 必須是另外再開一台機器執行嗎? 還是利用start-hbase.sh順便開啟就可以了?

2. 兩個不同的cluster是各自一個zookeeper cluster 還是共用一個?

3. 在配置做replication的過程,add_peer 'n',"zk.server.com:2181:/hbase"這個指令的zk.server.com是slave cluster的zookeeper 的ip位置嗎?

4. 當我在master replication執行start_replication,master cluster 的regionserver的log出現以下錯誤訊息

2013-11-10 12:46:40,404 WARN org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Waited too long for this file, considering dumping
2013-11-10 12:46:40,404 DEBUG org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Unable to open a reader, sleeping 1000 times 10
2013-11-10 12:46:44,981 DEBUG org.apache.hadoop.hbase.io.hfile.LruBlockCache: Stats: total=1.17 MB, free=248.44 MB, max=249.61 MB, blocks=3, accesses=237, hits=223, hitRatio=94.09%, , cachingAccesses=230, cachingHits=220, cachingHitsRatio=95.65%, , evictions=0, evicted=7, evictedPerRun=Infinity
2013-11-10 12:46:50,405 DEBUG org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: Opening log for replication cdclab2%2C60020%2C1384026404913.1384026419870 at 0
2013-11-10 12:46:50,565 WARN org.apache.hadoop.hbase.replication.regionserver.ReplicationSource: 1 Got:
java.io.EOFException
at java.io.DataInputStream.readFully(DataInputStream.java:180)
at java.io.DataInputStream.readFully(DataInputStream.java:152)
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1457)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1435)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1424)
at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1419)
at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.<init>(SequenceFileLogReader.java:55)
at org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.init(SequenceFileLogReader.java:177)
at org.apache.hadoop.hbase.regionserver.wal.HLog.getReader(HLog.java:742)
at org.apache.hadoop.hbase.replication.regionserver.ReplicationHLogReaderManager.openReader(ReplicationHLogReaderManager.java:67)
at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.openReader(ReplicationSource.java:563)
at org.apache.hadoop.hbase.replication.regionserver.ReplicationSource.run(ReplicationSource.java:369)

想請問哪位高手可以幫幫忙解說這錯誤的原因! 感激不盡!

發表人:  jazz [ 2013-11-12, 16:46 ]
文章主題 :  Re: HBase replication

nowitzki0205 寫:
1. zookeeper 必須是另外再開一台機器執行嗎? 還是利用start-hbase.sh順便開啟就可以了?
2. 兩個不同的cluster是各自一個zookeeper cluster 還是共用一個?


1. 就我理解,start-hbase.sh 在新的版本應該都有自動帶起 ZooKeeper,不過在 hbase-site.xml 必須開啟 hbase.cluster.distributed
代碼:
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>


2. 不同 Cluster (特別是異地)是各自獨立的 ZooKeeper Cluster

- Jazz

1 頁 (共 1 頁) 所有顯示的時間為 UTC + 8 小時
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group
http://www.phpbb.com/