Can i read hadoop file line by line
I wanna read line by line records in unix box from hadoop file system :
while read line do echo "input record " $line ###some other logic i have here.... done < /user/want/to/read/from/hadoop/part00
Above code snippet shows me error -
**: cannot open [No such file or directory]**
How can i read from Hadoop using Unix tools?
Use the hadoop fs command to access contents of these files:
while IFS= read -r line; do echo "Read: $line" done < <(hadoop fs -cat hdfs://nodename/filename)
Note that the <() construct requires bash; thus, your script needs to start with #!/bin/bash, not #!/bin/sh.