(+84) 463.28.7979

How to Clean/Remove Not Found Errors from Google web master tools generated from translated versions


I installed a translator plugin on one of my WordPress blogs but the plugin wasn’t working properly so I disabled it but two days later I found out that my Google web master tools account was reporting about 1100 ‘Not Found’ errors under the ‘Web crawl errors’ section. All the errors were from translated versions of my blog. I used the ‘robots.txt’ file to fix this issue.

If you don’t know what a ‘robots.txt’ file is, then read the article titled how to control access of the web crawlers or web robots to your site.

Basically, add rules to your ‘robots.txt’ file to Disallow any spider from indexing the translated version of the pages. My ‘robots.txt’ file looks like the following Depending on your situation you might need to block more languages. Just look in the Google webmaster tools and see which languages are causing the error then add them to the Disallow rule.


User-Agent: *
# Language pages
Disallow: /ar/*
Disallow: /bg/*
Disallow: /zh-hant/*
Disallow: /ca/*
Disallow: /cs/*
Disallow: /da/*
Disallow: /de/*
Disallow: /el/*
Disallow: /es/*
Disallow: /fi/*
Disallow: /fr/*
Disallow: /he/*
Disallow: /hi/*
Disallow: /hr/*
Disallow: /id/*
Disallow: /it/*
Disallow: /iw/*
Disallow: /ja/*
Disallow: /ko/*
Disallow: /lt/*
Disallow: /lv/*
Disallow: /mr/*
Disallow: /nl/*
Disallow: /no/*
Disallow: /pl/*
Disallow: /pt-br/*
Disallow: /pt/*
Disallow: /ro/*
Disallow: /ru/*
Disallow: /sk/*
Disallow: /sl/*
Disallow: /sr/*
Disallow: /sv/*
Disallow: /tl/*
Disallow: /tr/*
Disallow: /uk/*
Disallow: /vi/*
Disallow: /zh-CN/*
Allow: /

As far as I know, Google penalizes for duplicate content. Translated version of your page is considered duplicate content so for SEO benefit it is best to use this method to block access to the translated version of a web page.

It took about two weeks for all the errors to go away from my Google webmaster tools account but the number of errors started to go down as soon as I updated my robots.txt file to block the spiders from crawling all the translated version of the site. Hope this helps.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>