User-agent: Yandex Disallow: / User-agent: BaiDuSpider Disallow: / User-agent: Baiduspider+(+http://www.baidu.com/search/spider.htm) Disallow: / User-agent: * Crawl-delay: 6 Allow: / Disallow: /trackback/ Disallow: /cgi-bin/ Disallow: /wp-admin/ Disallow: /wp-login.php Disallow: /author/bl* Disallow: */feed Disallow: */feed/ Disallow: */trackback/ # Disallow all files with ? in url Disallow: /*?* Disallow: /*?