Get the total mapping and reducing times in hadoop programmatically

I am trying to calculate the individual total times of Mapping, Shuffling and Reducing by all tasks in my MR code.

I need help retrieving that information for each MapReduce Job.

Can someone post any code snippet that does that calculation?

Answers


You need to use the JobClient API as shown below: There are however some quirks to the API. Try it let me know i will help you out.

    JobClient client = null;

         Configuration configuration = new Configuration();

        configuration.set("mapred.job.tracker", jobTrackerURL);
        client = new JobClient(new JobConf(configuration));

        while (true) {


            List<JobStatus> jobEntries = getTrackerEntries(jobName,
                        client);


                    for (JobStatus jobStatus : jobEntries) {


                        JobID jobId = jobStatus.getJobID();
                        String trackerJobName = client.getJob(jobId)
                                .getJobName();
                        TaskReport[] mapReports = client
                                .getMapTaskReports(jobId);
                        TaskReport[] reduceReports = client
                                .getReduceTaskReports(jobId);
                        client.getJob(jobId).getJobStatus().getStartTime();


                        int jobMapper = mapReports.length;
                        mapNumber = +jobMapper;
                        int jobReducers = reduceReports.length;
                        reduceNumber = +jobReducers;


                    }
                }

Need Your Help

how to upload photo in react native android

android reactjs upload native

I use react native to develop in android. And I want upload a file or photo to server .And I use react-native-file-uploader,And I got errors:

Refresh screen after drawing a line in c#

c# windows screen line draw

I´m trying to write a simple application that prints a line "moving" at certain speed, it is my first time using c# and Windows to create an application, i found a tutorial that helps me drawing a ...