Why do Japanese people turn away and look elsewhere when you (foreigner) sit next to them on a train or bus?

I have lived in Japan for about half of my life. I have these situations happen to me to. To be honest I find that Japanese people are rather shy and don't find the eye contact thing a big plus. I have also ran into the people who will take my picture and stare me down when I am on the bus. It just happens. They think we are unique and some people think that as something cool and some are afraid of that. To be honest some will also stray because there english is really bad. Who cares though if someone is looking away or turn away because it is much better that the old man that will take your picture without you looking. FYI that is just strait up CREEPY!!