Tuesday, May 22, 2012

Hive Hbase integration/ Hive HbaseHandler : Common issues and resolution

It is common that when we try out hive hbase integration it leads to lots of unexpected errors even though hive and hbase are running individually without any issues. Some common issues are

1) jars not available
The following jars should be available on hive auxpath
  1. usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar
  2. /usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar
  3. /usr/lib/hive/lib/zookeeper-3.3.1.jar  
These jars vary with the hbase , hive and zookeeper version running on your cluster

2)zookeeper quorum not available for hive client
Should specify the zoo keeper server host names so that the leader hbase master server can be chosen for the hbase connection

hbase.zookeeper.quorum=zk1,zk2,zk3

Where zk1,zk2,zk3 should be the actual hostnames of the ZooKeeper servers


These values can be set either on your hive session or permanently on your hive configuration files

1) Setting in hive client session
$ hive -auxpath /usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar:/usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar:/usr/lib/hive/lib/zookeeper-3.3.1.jar -hiveconf hbase.zookeeper.quorum=zk1,zk2,zk3

2) Setting in hive-site.xml
<property>
<name>hive.aux.jars.path</name>
<value>file:///usr/lib/hive/lib/hive-hbase-handler-0.7.1-cdh3u2.jar,file:///usr/lib/hive/lib/hbase-0.90.4-cdh3u2.jar,file:///usr/lib/hive/lib/zookeeper-3.3.1.jar,file:///usr/lib/hive/lib/hive-contrib-0.7.1-cdh3u2.jar</value>
</property>


 <property>
<name>
hbase.zookeeper.quorum</name>
<value>
zk1,zk2,zk3</value>
</property>


Still hive hbase integration not working 
 
Error thrown : Master not running

Check whether the HbaseMaster is really down. If it is fine there could be other possibilities , a few common ones being
 
  • Firstly you need to check whether hbase.zookeeper.quorum is correctly set, it shouldn’t be localhost.
  • If multiple hbase clusters share the same zookeeper quorum then the znode parent value will be different for each. If that is the case, then set ‘zookeeper.znode.parent’ also has to be set in hive configuration, to the correct value from hbase-site.xml.


16 comments:

  1. Hbase is the Hadoop database. Think of it as a distributed, scalable, big data store.

    ReplyDelete
  2. I suppose I've selected an unbelievable and interesting blog. Big data analytics platform

    ReplyDelete
  3. Hadoop is really a good booming technology in coming days. And good path for the people who are looking for the Good Hikes and long career. We also provide Hadoop online training

    ReplyDelete
  4. Yes Hadoop is the buzzing technology in the present world.HBase is the data base used by Hadoop.One website named http://www.hadooponlinetutor.com is offering the hadoop complete videos at $20 only.Really the videos are good

    ReplyDelete
  5. In coming days hadoop is one of the best technology to migrate for all java people who ever interested and eager to get more income .
    Hadoop Tutorial

    ReplyDelete
  6. I was reading your blog this morning and noticed that you have a awesome resource page. I actually have a similar blog that might be helpful or useful to your audience.


    Regards
    sap sd and crm online training
    sap online tutorials
    sap sd tutorial
    sap sd training in ameerpet
    sap crm training tutorial

    ReplyDelete
  7. Thanks The information which you provided is very much useful for Pega Training v

    ReplyDelete
  8. Thanks The information which you provided is very much useful for Hybris Training v

    ReplyDelete
  9. Thank you,so thank of this tutorial! Hope you have more the lession like this!AWS Training
    Thank agains!

    ReplyDelete
  10. very nice article on Big Data. With the explosion of big data, companies are faced with data challenges in three different areas. hadoop online training

    ReplyDelete
  11. This comment has been removed by the author.

    ReplyDelete
  12. Our Complete in depth Java training course takes you to TOP Level IT companies with high end package. Arcus Offers Java J2EE real time training with placement assurance..
    java training in chennai

    ReplyDelete