shared connection in hadoop under setup method

I have initialized redis(jedis) client-connection in setup method in Hadoop MR job. When I use the connection in my map job to update the key,value pairs, Can the maps running on multiple nodes simultaneously use the same connection to update redis? Or is there a lock on the connection?

Answers


I am not sure if i got your question correctly. If in you case you have initialised the Redis client in the setup phase of a mapper then each mapper will use its own connection to communicate with Redis. Since all mappers run in their own JVM I don't believe you can share connections across them.

And yes,multiple clients can write to Redis simultaneously.If all them are writing to same key in Redis than it will internally handle this case but it never takes a cluster level lock.


If you have passed same connection session object to mapper then mapper will use that session to update redis. If this then there is no lock.


Need Your Help

Multivariable function operating on an array of vectors in R

arrays r function variables

I would like to define a multivariable function and then operate on an array of vectors, with the elements of the vectors the values of the variables. And, replace the array with an array of the o...

Netbeans 6.1 Incorrect CVS Status on a file that does not exist

netbeans cvs cvsnt

I have been trying to figure this out for a few hours off and on now and can't figure it out. I committed a lot of binary (jar files) to cvs and they worked fine, but one of the 6 directories, net...