User-agent: * Disallow: /*? Allow: /*?p= #Allow: /*?special_label=8463$ Disallow: /index.php/ Disallow: /wishlist Disallow: /admin/ Disallow: /sendfriend/ Disallow: /enable-cookies/ Disallow: /LICENSE.txt Disallow: /LICENSE.html Disallow: /skin/ Disallow: /js/ Disallow: /directory/ Allow: /*?utm* #Stop crawling user account and checkout pages Disallow: /*checkout/ Disallow: /*onestepcheckout/ Disallow: /*customer/ #Blocking native catalog and search pages: Disallow: /*catalogsearch/ Disallow: /*catalog/product_compare/ Disallow: /*catalog/category/view/ #Sometimes Webmasters block pages with filters.. Disallow: /*?dir* Disallow: /*?dir=desc Disallow: /*?dir=asc Disallow: /*?limit=all Disallow: /*?mode* #More reasonable to use canonical tag on these pages. #Blocking CMS directories. Disallow: /app/ Disallow: /bin/ Disallow: /dev/ Disallow: /lib/ Disallow: /phpserver/ Disallow: /pub/ #These commands are not necessary. Search engines are smart enough to avoid including CMS files in their index. #Blocking duplicate content: Disallow: /tag/ Disallow: /*/tag/ Disallow: /review/ Disallow: /it/shoponline Disallow: /it/review/product/listAjax/ Disallow: /it/catene-neve/*size= Disallow: /it/catene-neve/*year= Allow: /it/catene-neve/fiat/*year= Allow: /it/catene-neve/jeep/*year= Allow: /it/catene-neve/renault/*year= Allow: /*html?attribute151= Disallow: /it/barre-portatutto-portapacchi/*version=