Hadoop LZO & SnappyCodec error in Hadoop and Hive

I am using Ubuntu-12.04,Hadoop-1.0.2,Hive-0.10.0 while reading data about 1 million records from hive I got error below for query

select * from raw_pos limit 10000;
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

so I installed Snappy for Hadoop in $HADOOP_HOME/lib folder,which produces files libsnappy.a, libsnappy.la,libsnappy.so,libsnappy.so.1,libsnappy.so.1.1.4

also add hadoop-lzo-0.4.3.jar in $HADOOP_HOME/lib/ & make changes in cor-site.xml,mapred-site.xml as follow

Core-site.xml:-

<property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:54310</value>
</property>

<property>
    <name>hadoop.tmp.dir</name>
    <value>/home/hadoop/apache/hadoop-1.0.4/hadoop_temp/</value>
    <description>A base for other temporary directories.</description>
</property>

<property>
<name>io.compression.codecs</name>
<value>
  org.apache.hadoop.io.compress.SnappyCodec
</value>

mapred-site.xml :-

<property>
    <name>mapred.job.tracker</name>
    <value>hdfs://localhost:54311</value>
</property>

<property>
  <name>mapred.compress.map.output</name>
 <value>true</value>
</property>

<property>
   <name>mapred.map.output.compression.codec</name>
   <value>org.apache.hadoop.io.compress.SnappyCodec</value>
</property>

but when I started hive & do show databases, gives error:-

Failed with exception java.io.IOException:java.io.IOException: Cannot create an instance of InputFormat class org.apache.hadoop.mapred.TextInputFormat as specified in mapredWork!

Answers


Modify your core-site.xml to this and see if it helps :

<property>
<name>io.compression.codecs</name>
<value>com.hadoop.compression.lzo.LzoCodec,com.hadoop.compression.lzo.LzopCodec,org.apache.hadoop.io.compress.SnappyCodec</value>
</property>

<property>
<name>io.compression.codec.lzo.class</name>
<value>com.hadoop.compression.lzo.LzoCodec</value>
</property>

Need Your Help

Bootstrap 3 form submission

javascript jquery html forms twitter-bootstrap

I'm using bootstrap 3 to build a contact form and I have the following problem: If submitted empty, the form submits to the same page. If i enter values in the boxes, it will redirect to the homepa...

Arduino serial data parsing

arduino

I'm writing an app to control my robot with my Android phone over Bluetooth, everything is goes well, data is echoed and verified, but I'm having some trouble with the protocol, specifically I want...