shared connection in hadoop under setup method
I have initialized redis(jedis) client-connection in setup method in Hadoop MR job. When I use the connection in my map job to update the key,value pairs, Can the maps running on multiple nodes simultaneously use the same connection to update redis? Or is there a lock on the connection?
I am not sure if i got your question correctly. If in you case you have initialised the Redis client in the setup phase of a mapper then each mapper will use its own connection to communicate with Redis. Since all mappers run in their own JVM I don't believe you can share connections across them.
And yes,multiple clients can write to Redis simultaneously.If all them are writing to same key in Redis than it will internally handle this case but it never takes a cluster level lock.
If you have passed same connection session object to mapper then mapper will use that session to update redis. If this then there is no lock.