Are You Trying to Hide From Google?

July 24th, 2008 Amir Posted in Website tips | No Comments »

The way language selectors are implemented in multi-lingual websites determines their search engine visibility.

Search engines will serve what they can find and index. If your website exists in multiple languages, you’d probably want the translated pages to show up in search results, right? The easiest way for search engines to notice your translated pages is through the language selectors. Here are a few examples:

: Language selectors as text links

Text links selectors

These language selectors are implemented as simple text links. The links leads to the translated contents.
The HTML code for each link is just:

<a href="translated_url.html">Russian</a>

Drop down languages list

Drop down language menu

The language selection is implemented as a drop down menu. It’s compact and convenient, but completely useless for search engines. Here’s the HTML code to implement it:

<FORM>
<SELECT onChange="load_page(this)" align="RIGHT">
<OPTION SELECTED VALUE="index.html">English</OPTION>
<OPTION VALUE="ru/index.html">Russian</OPTION>
<OPTION VALUE="de/index.html">German</OPTION>
<OPTION VALUE="zh/index.html">Chinese</OPTION>
<OPTION VALUE="es/index.html">Spanish</OPTION>
</SELECT>
</FORM>

Quick quiz: What problems do you see with this?

It’s a form – search engines don’t submit forms so they’re just going to ignore this one.
There’s no submit button – even if Google decided to try and follow this form, it doesn’t have a submit button. The language is selected using a Javascript call (load_page).

Basically, Google and other search engines are just not going to see the translated pages using this drop down selector. If you decide to implement something like this, make sure you give search engines another way of reaching the translated pages. A site map would be a good idea (both as an HTML page and as a Google sitemap).

The worse option – using a cookie to select the language

Supposing your website is rendered using ASP, PHP or any other server side code, you can easily use session cookies to determine which language to serve. It’s very simple to implement:

The language selection link or form sets the language value in the session.
Every time a page is rendered the server checks that session variable and serves the correct contents.

A webmaster’s dream – you implement it in one place and it works for the entire website. A day’s work in 10 minutes!

So, what kind of problems might we face by implementing language selection in session cookies?

No matter what we do, only one language is going to appear in search results – forever.

Since each URL can serve multiple contents, depending on a cookie, the rest of the contents are never going to appear. At best, you’ll get just your default language. If Google manages to see you’re service multiple contents for the same URL, most changes are nothing is going to be indexed (or ever appear in search results).

How is my site doing?

To check our your multi-lingual website is doing in search engines, you can just ask them

Search Google for what pages it scanned in your website

site:www.mywebsite.com

If a page doesn’t appear in this list, it’s not indexed and will not appear in search results.

Check if a page is cached

cache:www.mywebsite.com/products.html

You’ll see if this page has been scanned and when. If it’s not scanned, it doesn’t appear in search results.

Check the page’s rank

Use Google’s pagerank (available from the Google toolbar). If the page rank is way lower than the rank of other pages in your website, it means Google managed to find it, but through a long chain of links. It knows it’s there but it doesn’t consider it a very important page.