Why do Japanese people turn away and look elsewhere when you (foreigner) sit next to them on a train or bus?

In Japan it is considered rude to stare at people. Japan is very busy so most of the time they'll be talking on the phone or thinking about their jobs or test coming up at school. Don't take it the wrong way, in fact look at this experience as knowing that they are respecting you as they would if you were originally Japanese.