欢迎关注Hadoop、Spark、Flink、Hive、Hbase、Flume等大数据资料分享微信公共账号:iteblog_hadoop
  1. 文章总数:961
  2. 浏览总数:11,490,360
  3. 评论:3873
  4. 分类目录:103 个
  5. 注册用户数:5846
  6. 最后更新:2018年10月17日
过往记忆博客公众号iteblog_hadoop
欢迎关注微信公众号:
iteblog_hadoop
大数据技术博客公众号bigdata_ai
大数据猿:
bigdata_ai

Hbase 0.96.0分布式安装手册

  在《Zookeeper 3.4.5分布式安装手册》《Hadoop2.2.0完全分布式集群平台安装与设置》文章中,我们已经详细地介绍了如何搭建分布式的Zookeeper、Hadoop等平台,今天,我们来说说如何在Hadoop、Zookeeper上搭建完全分布式的Hbase平台。这里同样选择目前最新版的Hbase用于配合最新版的Hadoop-2.2.0,所以我们选择了Hbase-0.96.0。
  1、下载并解压Hbase:

[wyp@master Downloads]$ wget http://mirror.bit.edu.cn/apache
                             /hbase/hbase-0.96.0/hbase-0.96.0-hadoop2-bin.tar.gz
[wyp@master Downloads]$ tar -zxvf hbase-0.96.0-hadoop2-bin.tar.gz
[wyp@master Downloads]$ cd hbase-0.96.0-hadoop2-bin/
[wyp@master hbase-0.96.0-hadoop2]$

  2、修改Hbase的一些默认环境配置:

[wyp@master Downloads]$ vim conf/hbase-env.sh  
# The java implementation to use.  Java 1.6 required.
export JAVA_HOME=/home/wyp/Downloads/jdk1.7.0_45
(Java的home目录得根据你自己机器设定,如果不知道你机器的
Java home,可以运行which java查看)

# Tell HBase whether it should manage it's own instance of Zookeeper or not.
export HBASE_MANAGES_ZK=false
HBASE_MANAGES_ZK默认是true,说明是让Hbase 单独管理 zookeeper。如果你机器上已经安装好了分布式Zookeeper,那么可以设定为fals;否则就不需要修改。

  3、修改hbase-site.xml文件,并在里面添加如下的配置:

[wyp@master Downloads]$ vim conf/hbase-site.xml
<property>  
    <name>hbase.rootdir</name>  
    <value>hdfs://master:8020/hbase</value>  
</property>  
  
<property>  
    <name>hbase.master</name>  
    <value>hdfs://master:60000</value>  
</property>  
  
<property>  
    <name>hbase.cluster.distributed</name>  
    <value>true</value>  
</property>  
  
<property>  
    <name>hbase.zookeeper.property.clientPort</name>  
    <value>2181</value>  
</property>  
  
<property>  
    <name>hbase.zookeeper.quorum</name>  
    <value>master,node1,node2</value>  
</property>  
  
<property>  
    <name>hbase.zookeeper.property.dataDir</name>  
    <value>/home/wyp/zookeeper</value>  
</property>  
  
<property>  
    <name>hbase.client.scanner.caching</name>  
    <value>200</value>  
</property>  
  
<property>  
    <name>hbase.balancer.period</name>  
    <value>300000</value>  
</property>  
  
<property>  
    <name>hbase.client.write.buffer</name>  
    <value>10485760</value>  
</property>  
  
<property>  
    <name>hbase.hregion.majorcompaction</name>  
    <value>7200000</value>  
</property>  
  
<property>  
    <name>hbase.hregion.max.filesize</name>  
    <value>67108864</value> 
</property>  
  
<property>  
    <name>hbase.hregion.memstore.flush.size</name>  
    <value>1048576</value>  
</property>  
  
<property>  
    <name>hbase.server.thread.wakefrequency</name>  
    <value>30000</value>  
</property>  
  请注意上面的hbase.rootdir、hbase.zookeeper.quorum以及hbase.zookeeper.property.dataDir的配置;其中hbase.rootdir对应Hadoop的$HADOOP_HOME/etc/hadoop/core-site.xml文件中fs.defaultFS的值(可以参见本博客的《Hadoop2.2.0完全分布式集群平台安装与设置》文章);hbase.zookeeper.quorum的值就是Zookeeper配置的几台机器的hostname,可以参见本博客的《Zookeeper 3.4.5分布式安装手册》;hbase.zookeeper.property.dataDir对应Zookeeper的$ZOOKEEPER/conf/zoo.cfg文件中dataDir属性的值。

  4、修改regionservers 文件配置,这个文件是用来指定regionserver的机器的:

[wyp@master conf]$ vim regionservers

在里面添加如下内容,一台机器名一行
到底需要选择什么机器作为regionserver可以根据你自己的情况来决定
master
node1
node2

  5、同样将配置好的hbase打包,分发到node1、node2中:

[wyp@master Downloads]$ tar -zcf hbase.tar.gz hbase-0.96.0-hadoop2
[wyp@master Downloads]$ scp hbase.tar.gz node1:/home/wyp/Downloads/
[wyp@master Downloads]$ scp hbase.tar.gz node2:/home/wyp/Downloads/

  6、分别登录到node1、node2解压刚刚发送的hbase.tar.gz文件:

[wyp@master Downloads]$ ssh node1
Last login: Sun Jan 19 15:17:46 2014 from master
[wyp@node1 ~]$ tar -zxf Downloads/hbase.tar.gz
[wyp@node1 ~]$ exit
logout
Connection to node1 closed.
[wyp@master Downloads]$ 

  7、分别启动master、regionserver等进程:


在master启动master进程
[wyp@master Downloads]$ $HBASE_HOME/bin/hbase-daemon.sh start master

在node1、node2中启动master进程
[wyp@node1 Downloads]$ $HBASE_HOME/bin/hbase-daemon.sh start regionserver

  8、检查Hbase是否成功部署
打开http://master:60010,如果能进去,说明安装成功!

在启动master进程的时候,可能会出现以下的异常信息:

org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ipc.RpcServerException): 
                          Unknown out of band call #-2147483647
        at org.apache.hadoop.ipc.Client.call(Client.java:1347)
        at org.apache.hadoop.ipc.Client.call(Client.java:1300)
         ...................................
        at com.sun.proxy.$Proxy13.getFileInfo(Unknown Source)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:606)
         ...................................
        at com.sun.proxy.$Proxy13.getFileInfo(Unknown Source)

这是因为Hbase0.96中所用的Hadoop jar包和Hadoop 2.2.0中的相关Jar包不一致,请用Hadoop-2.2.0中的share/hadoop文件夹下面的jar包替换Hbase中lib下面的相应包。然后再重启Hbase即可。

本博客文章除特别声明,全部都是原创!
转载本文请加上:转载自过往记忆(https://www.iteblog.com/)
本文链接: 【Hbase 0.96.0分布式安装手册】(https://www.iteblog.com/archives/902.html)
喜欢 (1)
分享 (0)
发表我的评论
取消评论

表情
本博客评论系统带有自动识别垃圾评论功能,请写一些有意义的评论,谢谢!
(6)个小伙伴在吐槽
  1. 我有两点不是很明白:1、Hbase只需要一个master就行了吗?那万一这个master挂了,该怎么办呢?2、你上面已经说了不使用hbase来管理zookeeper,那么为什么在hbase-site.xml文件中还要设置hbase.zookeeper.property.dataDir 及hbase.zookeeper.quorum。这些都是在zookeeper中配置的。我现在已经使用QJM+ZOOKEEPER实现了HDFS的高可用,现在在安装Hbase,也要用到zookeeper,所以就想用之前的zookeeper。以上的两点就是我不明白的。希望你能给我解解惑!谢谢!!
    Bob2014-09-23 17:22 回复
    • 1、至少到Hbase 0.96还是只有一个master,不过到了后期版本貌似有相应的措施来处理master单点故障。你可以去看下最新版本的hbase。2、在hbase-site.xml里面配置那个是告诉Hbase找到你自己配置好的Zookeeper。
      w3970907702014-09-23 17:43 回复
      • 多谢!
        Bob2014-09-24 09:59 回复
  2. 初次学习hbase,一直出现下面的错误????[code]2014-05-26 16:58:40,830 INFO org.apache.hadoop.hbase.master.ActiveMasterManager: Master=cloud001,60000,14010947195042014-05-26 16:58:41,097 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception. Starting shutdown.org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4 at org.apache.hadoop.ipc.Client.call(Client.java:1070) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225) at com.sun.proxy.$Proxy10.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:379) at org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:119) at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:238) at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:203) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:89) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1386) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1404) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:254) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:187) at org.apache.hadoop.hbase.util.FSUtils.getRootDir(FSUtils.java:666) at org.apache.hadoop.hbase.master.MasterFileSystem.(MasterFileSystem.java:112) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:573) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:433) at java.lang.Thread.run(Thread.java:744)2014-05-26 16:58:41,105 INFO org.apache.hadoop.hbase.master.HMaster: Aborting2014-05-26 16:58:41,105 DEBUG org.apache.hadoop.hbase.master.HMaster: Stopping service threads2014-05-26 16:58:41,105 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server on 600002014-05-26 16:58:41,105 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 0 on 60000: exiting2014-05-26 16:58:41,105 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 0 on 60000: exiting2014-05-26 16:58:41,105 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 2 on 60000: exiting2014-05-26 16:58:41,105 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC Server handler 1 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 1 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 2 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 3 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 4 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server listener on 600002014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 6 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 7 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 8 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 9 on 60000: exiting2014-05-26 16:58:41,106 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server handler 5 on 60000: exiting2014-05-26 16:58:41,108 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder2014-05-26 16:58:41,108 INFO org.apache.hadoop.ipc.HBaseServer: Stopping IPC Server Responder2014-05-26 16:58:41,108 INFO org.apache.hadoop.hbase.master.HMaster: Stopping infoServer2014-05-26 16:58:41,109 INFO org.mortbay.log: Stopped SelectChannelConnector@cloud001:600102014-05-26 16:58:41,303 INFO org.apache.zookeeper.ZooKeeper: Session: 0x46378ee95d0003 closed2014-05-26 16:58:41,303 INFO org.apache.hadoop.hbase.master.HMaster: HMaster main thread exiting2014-05-26 16:58:41,303 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down2014-05-26 16:58:41,303 ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start masterjava.lang.RuntimeException: HMaster Aborted at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:160) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2129)[/code]
    xuhui322014-05-26 16:58 回复
    • [code]org.apache.hadoop.ipc.RemoteException: Server IPC version 9 cannot communicate with client version 4[/code]你看下这个错误,是版本问题,你的Hbase、Hadoop版本分别是多少?
      w3970907702014-05-27 10:41 回复