Like most travellers who visited Japan the first time, my first visit to Japan was memorable. At that time I was not able to read any Kanji, Hiragana or Katakana. I was struggling to figure out where the toilets are, where the bus stop and train station are, and even having trouble talking to people since many Japanese do not understand English. But do not worry, my first visit to Japan was 13 years ago! Japan has changed a lot since then.
You will see more English letters on public places and you have no worries finding where the toilets, bus stop, taxi or train station are because they all have signs, such as red/pink for women toilet and blue/black for men, a yellow signboard with a black shape bus, taxi and train... like those ones you probably have seen in other countries. First, what about taking an online visit to major international airports here in Japan and get familiar with the surroundings?
http://www.narita-airport.jp/en/index.html
Or if you arrive in Haneda, check out:
http://www.tokyo-airport-bldg.co.jp/en/
Or arriving Kansai
http://www.kansai-airport.or.jp/en/access/
Young Japanese people nowadays learn English and they will be very happy to tell you the ways. My suggestion is to ask for help from the right person at the right timing. For example, if you are in trouble during rush hours, do not ask those people who are commuting, since you'll probably get ignored. In fact it is best not to walk around for sightseeing during rush hours (7:30-8:30), especially in major cities such as Tokyo, Shinjuku, Shibuya, Ikebukuro, Kyoto, Nagoya, .... the list can go on.
For best shots of Tokyo station (daylight and daytime), check out this homepage (you may not read the Japanese letters, but you can click on the photos 800x600 – ref. size of photo):
http://www.marunouchi-photo.com/tokyosta/
What to see in Tokyo:
http://www.tokyotower.co.jp/english/
... as for me i love Tokyo Tower at night (Rainbow color is available from February from 8pm to 11 pm), viewing it from another building...
... most five star hotels in Tokyo will have a nice view of Tokyo Tower, but i recommend the cheapest way, take a train to Machida station, walk towards Keio University, and at Sakurada crossroad you get to see full picture of Tokyo Tower....
... do you know that Tokyo Tower was built to resemble the Eifel Tower in Paris? ....
..back to language barriers:
Having said that you need not worry about language barriers, you do want to get prepared when travelling in a foreign country, whose national language is not English. For a quick Japanese lessons check out my earlier note on My First Trip To Japan.
(most of these words you probably already know by now):
When in trouble, before you ask for help, it is best to say:
“....ano....sumimasen....”
To find out where “xxxx” is, you then say:
“xxxx wa doko desuka? Oshiete kudasai”
When the person is helpful, regardless whether his/her info is useful, you then say:
“arigato”
But when you have no idea at all what he said (even with the sign language), you then say:
“...sumimasen,.... wakarimasen....” (a sign language that could help translate your lack of understanding is to scratch your head – looking a little confused and helpless)
Japanese use a lot of sign language even when they speak to their own people. Body language is in fact very much used in Japanese communication.
One of my cultural shocks during my first few weeks in Japan was when I visited shops, stores or restaurants, shopkeepers and waitresses greet me so politely that I felt a little comfortable at first. Yes, here customer service is a priority to all businesses and you will get these “king and queen” treatment for free (no tipping is necessary at all in Japan).
Here you may get to spend some money while having the joy of being “served for free”. Language barriers should not stop you from visiting Japan, the most important topics should be "where should be spending your Yen to get the most of your visit?"
When you arrive at 
