User-agent: Mediapartners-Google* Disallow: ## User-agent: * ## Disallow: ## ## Supposedly this will allow all robots to crawl the ## entire site. Temptation is to disallow unlinked ## directories/pages. But experience shows that this ## merely encourages crackers. You know who. Better ## to leave tham all unnamed. ##