Moving huge local files to HDFS using java

I have about 20 millions files stored on my local file system, each file 5k represents a tweet.

This stored as the following:


Example1 : /home/username/tweets/SCP/2014/04/11/9989443342233.txt

Example1 : /home/username/tweets/WDR/2014/02/08/5890321764568.txt

So is it possible to write a map reduce java program to move all tweets under a certain tag to a singe directory in HDFS based on the tag.

Any similar examples?


As seen in,

Make the sequencefile first, then upload it to HDFS.

Need Your Help

Add sql database in mfc dialog based application

visual-c++ mfc

I have a MFC dialog based project,Now I want to add database with that project.

Panel Open close not working

javascript jquery html css jquery-mobile

Hi I have requirement where Jquery mobile panel will be opening when some conditions are met i have tried close,open panel but that was not working. Please help Thanks in advance