Cannot run Hive metastore with remote mode

I added Hive in my Apache Hadoop distributed cluster. Instead creating metastore in local directory, I would like to create metastore in HDFS. However, my settings in hive-site.xml don't seem to work. I got the error like below, which seemed that hive still tried to use run metastore in local mode.

java.sql.SQLException: Directory /home/zz/metastore_db cannot be created.

Can anyone tell me what has gone wrong with my settings? Thanks a lot!

Below is my hive-site.xml content:

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://client2/metastore</value>
</property>

<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>MySQL JDBC driver class</description>
</property>

<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
</property>

<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>user name for connecting to mysql server </description>
</property>

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hadoop</value>
</property>

<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
</property>

<property>
    <name>hive.server2.servermode</name>
    <value>thrift</value>
</property>

<property>
    <name>datanucleus.autoCreateSchema</name>
    <value>false</value>
</property>

<property>
    <name>datanucleus.fixedDatastore</name>
    <value>true</value>
</property>

<property>
    <name>hbase.zookeeper.quorum</name>
    <value>master1</value>
</property>

<property>
    <name>hive.metastore.uris</name>
    <value>thrift://client2:9083</value>
    <description>IP address (or fully-qualified domain name) and port of the metastore host</description>
</property>

Answers


Make sure hive-site.xml file is there in hive classpath. In your case the updated hive-site.xml file is not taking properly instead it takes default values. For explicitly setting hive conf directory following environment variable can be used, before executing hive command

export HIVE_CONF_DIR=<PATH-TO-CONF-DIR>  

(directory contains hive-site.xml should be specified )

For configuring remote mysql as hive metastore, First you got to create the database with enough privileges in remote mysql(metastore database). Initially the database will be empty, So by setting the following property to true will automatically create the metastore schema in the remote db, if the database is empty

<property>
    <name>datanucleus.autoCreateSchema</name>
    <value>true</value>
</property>

If the issue still persists start hive using the option --config as follows

hive --config <PATH-TO-CONF-DIR>

Need Your Help

Passing Javascript value to C# Code Behind

c# javascript jquery html asp.net

I've coded myself into a bit of a corner, and the suggestions I've come across don't seem to be helping.

Java string encoding conversion within a webpage

java encoding utf8-decode windows-1255

I have a webpage that is encoded (through its header) as WIN-1255.