Can i read hadoop file line by line

I wanna read line by line records in unix box from hadoop file system :

Example -

while read line

do 

echo "input record " $line
###some other logic i have here....
done < /user/want/to/read/from/hadoop/part00

Above code snippet shows me error -

**: cannot open [No such file or directory]**

How can i read from Hadoop using Unix tools?

Answers


Use the hadoop fs command to access contents of these files:

while IFS= read -r line; do
  echo "Read: $line"
done < <(hadoop fs -cat hdfs://nodename/filename)

Note that the <() construct requires bash; thus, your script needs to start with #!/bin/bash, not #!/bin/sh.


Need Your Help

How to pass a value from Cocoa to an SQLite query

objective-c cocoa sqlite nsstring

first of all please let me say that I am quite new to objective c development. I am writing a small app for personal use for the iphone, but I have some problems executing the following code:

ShareKit Connect with Facebook

ios facebook sharekit

I have an application where users need to login/create an account in order to use it. I am trying to implement Connect with Facebook through ShareKit 2.0. Do you know if this is possible and can you