FoxyProxy, Hadoop, and SOCKS

Posted on June 03, 2008

FoxyProxy makes it very easy to talk to your Hadoop cluster running on EC2.

Run ssh with the -D command:

ssh -D 2324 ec2-75-101-XXX-XX.compute-1.amazonaws.com

Tell FoxyProxy to “use SOCKS proxy for DNS lookups” (tools > foxyproxy > more > global settings > use SOCKS proxy for DNS lookups)

Configure foxyproxy with rules for when to use local port 2324. Use wildcards like httpec2internal*.

All the features I cared about worked when set up this way.

(And of course the choice of 2324 isn’t special - use any port you like.)

Here’s a screenshot of the the proxy settings I use. It’s been a while since I configured it - I suspect that at least one of these lines is obsolete:

screenshot