HTTPS Request in Mapreduce
I'm trying to make an HTTPS request from the Mapper class of my job. I'm using Java's HttpsURLConnection library, which requires me to set the following properties: javax.net.ssl.keyStore javax.net.ssl.keyStorePassword
There's no problem setting these properties in a Java application, but in MapReduce I'm having trouble setting these properties at the mapper level. I can't figure out how to "reach outside" of Hadoop and set these properties for every mapper.
It is not a good approach to create request from MapReduce. The intuition is: you use MapReduce to be able to scale your computation. You havent provided more info about what you are doing, so lets think about what happens when you would scale (orders of magnitude) the task.
If your goal is to connect to one place, you are effectively (D)DoSing it. The server on the other end would not probably scale up as well as your MapReduce (cluster).
If you are crawling many targets, consider using some crawling distributed framework (find some on Google).
Solution wasn't related to anything with the application code, but rather cluster configuration. The keystore file needed to be placed in the same location on each data node. Not exactly endorsing this as the safest solution in terms of security, but it worked for a quick POC.